ED ^109 U7;5 



DOCOHEHT' RESDHE 



CE :00a_3,32 



TITLE 



INSTITOTION 
SPONS AGENCY. 

PDB DATE 
CONTRACT 
NOTE 

EDRS ?^ICE 
DFSCRIPTOI^S 



IDENTIFIEES*, 



An- Evaluation of Vocational Exemplary Projects.; Pirt 
D Vocational Education Act Amendments of 1968; ; 
Executive Summary and Final pepott, • 
Development A^ssociates^ Inc., Washington^ D.C* 
Office ^of Education • (DHEH) ^ ,Hashin^tonr« D.C. ' Of f ice 

'.'Of Planniti^V BudV^ting; an'^b Svaluation. 
Mar 75 " 

. ^EC-0-73-f'6663 

186p. * , . 

MF-$0.76 HC-'Sg.SI PLUS POSTAGE 
, Career Education; Cooperative Programs; Xost 

Effectiveness; ^Educational Legislation;' Educational' 
Programs; Elementary Secondary Education; '.Fedeiral 
Legislation'; Federal Programs; *National -Surveys; . 
♦Pilot Projects; Program Cost:3; *Program .Evaluation;' 
^ School Indu^stry Relationship; 'state Progr,am^; ' Tablea 
^ (Data).; *Vocation4l Educations . ' ' ' 

, Vocational- \Education^ Act of 196,3; Vocational 
Education.' Amendments of 1968 



AfesTfEACT" . .^'[J^ / y,,.' ' 

'/ 'Th€'«s:iudy evaluates the/effectiveness .of 50 projects 
^di^tRi^i!sj:,^iB^ \\iTi^^T .?^r!€-D of €h'^\19^8 ^inen.dments t^^ the VocXt ional;/^ 
- EdflCaiii^n 3^<?t of- 1^il'f,i(oiie for .ekoh St4t^.^ except .'fiawaiii and for /the 
'j)istrio^{ oi- OolViM'a^^ t-P-, afes'isV ,st^udents in bbtaTnina v - 

' s^^ipiix^kii^XoY^Q-i^^'^ aHy .^>'6'3'2 'par'ticii^^'ting and' 



aining 
(f03' 




, gfr^4t^i^3t *ii?p5ifet '(^^ Eart prpg^ams: 'was at .the >*l,e;fieat^ry ' levels and 
"that.; ;-on"a^ p-iroi^cf.-by-pro'jeci basi-s^ ! the 'pi:ogra|n^> ,im^acf oix studeritsl . 
Has..sntall*/.H«'4a'r<ix^g ihe y;eiatiaps,iiip. of ^ost ..to. student outcpmes^> 'it\ 
yas\.f^UT)^;^'ti?»<at] pp6j^^ pqsitive./ef f ects on. students ^ 

yhere ' r.ei#LtiV^^ fiind^ i^ere^exp-^nded p.n relatively fewef 

- 's^ud^i;jt'4v "^h^^^^ failure o^ student level v^l^ 

outi?'oio^S^ of ' t>,^/.p)fograjns to" ^ieet the level- planned most likejl.y - / . 
.oe<i:tirrea.vfe^ca'a5e of .£lx-e jgeTier'al Tack of clearly define.d ob^eativ'es^ ; 

ahd 




uifed^ by ;^b|jC:'/^^^ jnan^ 4^^^^'^^^ ^^P^^'^A?^^^ \ - 

ilable ,fro^^ o^Ker 's6utqes\ E^RtC^.rtake^^ every effo;;^t-^* 
t copy available', tiey^rtlielj^s;?.. Xtefcs of 'marginal 
re often e-nc'bunter^'d an:d tM^'^Mpts, the quality 



' * ^ J)<)C'Ume at s ^u^guir ed 
n*/®at^ria Is .not. available ,f fo^m 
" to obtain the besi 
. ♦ ^producibility" ai 
, * of, the microfiche 

* via the ERIC Doci 

'.*>rest>onsible for — ^ ^ ^- — , — v /- • / ^- , ^ . 

* supplied by EDfiS are'the best that '«liVbe tade .fror th? or,iginaI*,f */' 



LTV 

o> 

o 

o 



.vV , •'-V Vv\ I ' /''Ce y A L U A T I O N OF 



;^t?.CA^i:ij[.?^A^'L.- ^PXEMPLARY PROJ ECTS' 
^ ' «Pa r.«Jr^p^. V ^cabi'?n<^.l Education Ac-t ^ 



, * 



:*: w 'Pe^J tmeht' of i-iealth^ 'Kduc-ation, abci- W^jlfar^ 



' NATIONAL IWTtVutf^jif-' - " 

- Z .^PUCAflON' V. 

" ' .t^lS- OQCUAAENT- 'was BetN* REP(?0- 

. - ' ^'"J-«^-^E'*^50^9Rt>5j&ANJZ AT ION ORIGIN. 
- ^Ar»F5G (Uf»CnN>^ OF-VIEW OR OPINIONS 

^/ / rA.rtt> 00 NOT ne(;essarily repre 



. .^^i'^.y-^^t^NJ Of P^KiAl.fiA-T??)N'AL INSTlTUTEQF 

, . ^' ' -'WN5DueAriON,Pj05iT;?9N- OR POLICY 



Submitted ^By^ ' ' . 
.I>I?-ygLaPjyl5NT/ASS"d^^ INC. 
Washington, Di^C^J ''.20036 - . '7; 





The work reporhed herein was performed pursuant, to a contract with the United 

Sta^tes Department of Health, Education, and Welfare^ Office of Education. The 

statements, findings, conclusions, recommendations, and other data in this 

report are solely those of the Contractor and do not necessarily reflect the views 

of the Office of Education. / , / V \ 



t 




ACKNOWLEDGEMENTS 



Development Associates project staff would like to express their appreciation to 
the staff of the U.S. Office of Education's Office^ai Planning, Budgeting, and g 
p:valuation, and the Division of Vocational and Technical Education for their 
assistance in carrying out this project. Our appreciation is also extended to the 
i-tate officials., and most impjrtantly to the s-chool staff at each of tft^schools we 
M-itffl. With ml their c-joper^t^iftn. Development Associates' project st^ could 
n >t have accomplished this stucly' s 'field work. ' 

We^ari' indebted to the f-jUowing ,p roject field staff, technical and'suppo^-t staff 
t')r many 1 -ivj ]\ Aix^b and for excellent and faithful performance. 

Field Staff 



Tomas Torres John Gutierrez 

Iy)uis *Galvez Salvador Chavez 

Daniel Trevino John Plakos 

^, ^ Carlos Otal - ^ George Spicely 

Ricardo Lucero ^ f 

Technical Support 

Stanley Lichtenst§in,-- Eddie A. Taylor * 
Douglas Pepin * " ' Bruce Zweig \\ 

* * * " Howard Fleischman 

Supp.orV^ Staff 

Jean Rogers Kathleen Gardiner- 

T ^ Susan Shimer ^ . 

\ 

We are particularly indebted to teeter B. Davis, Senior Vice President of DA and 
officer-in-charge of this study, for wise counsel, leadership, and many hours 
reading and editing the draft of this -study. ^• 

4 Malcolm Young, Project Director 

' Russell Schuh, Asst. Project' Director 

Washington, D.C. 
March 1975 



EXECUTIVE SUMMARY 

Final Report on- Evaluation of Vocational Exemplary Projects; Part D, Vocational \ 
' , Education Act Amendments of J968 \ 

t 

Contractor: , Contract Number: 

Development Associates, Inc. OEC-0- 73-6663 

1521 New Hampshire Avenue, N. W^. 

Washington, 'D. C. 20036 ' ' March 1975 

I. PURPOSE OF THP STUDY 

Description of the Program: ^ " * ^ * 

Part D of the 1968 amendments to the Vocational Education Act of 1963 provides 
for the funding uf exemplary projects which represent new ways to create ct bridge 
between school and earning a living for young people. Half of the funds appro- 
priated for Part D are administered directly by the federal government through the 
Division of Vocational and Technical Education (DVTE) in the U:S, Office of Educa- > 
tion and half are administered by the states and territories. 

Between June 1970 and July 1972, DVTE awarded^rants averaging approximately 
$120,000 a year for three years to institutions in each state and the District of ' - 

Columbia to administer projects in accordance with a USOE policy paper (AVL \ 
V70-1), The policy paper provided that each project would ccjdinbine in one opera- 
tional setting all of the following: 

1, Provision for broad occupational orientation at, the elementary and secondary 
school levei^ so as to increase student awarene&s of the range of options open 
to them in the world of work", \ 

\ . , ^ ■ ' 

2, Provision for work experience, cooperative education, and similar programs, 
making possible a wide variety of offerings in many occupational areas, ^ ^ ' 



3, Provision for students not previously enroUed in vocational programs to 
receive specific training in job^ntry skills .just prior to the time that they 
leave the school. (Some of these tracing programs rnight b^ very intensive 
and of short duration, ) ^ 

4, Provision for intensive occupatibi^ guidance and counseling during the last 
years of school and for initial plaBment of all students^at the completion of 
therr schooling, (Placement might be in a job or in post-secondary occ.upa- 
tional tj;aining.'^ Placement should^ be accomplished in cooperation with appro- 
priate ebnploym en t services, manpower agencies, etc.) 

5, Provision for the grantee ot contractor to carry the program on with support 
from regular funding Sources after the termination of the federal assistance 

^/t /TT ^ under Part D of P, L. 90-576, (Federal assistance under Part D cannot exceed ^ 

three years,*) ' - - ^ 

The projects funded during this period constituted the first round of Part D fij^nding, 
and are considered by many to be the earliest federally funded efforts to implement 
on a comprehensive basis the concept of career education, S/nce 1972, a second 
round of projects guided by a modified statement of USOE policy has been funded, ^ 
In FY '76, a new set of grants to support a thirti round of projects following a new / 
set of policy guidelines is planned. > ^ 

' ' - 1 - ^ . " ^ 



Reason for the Study: 



SI the 



The primary purpose o| the study wa^ to evaluate the effectiveness 'of the first 
round of federally administered Part D projects and their components in terms of 
the extent \o which student outcomes attributable to project activities conformed to 
the legislative intent for Part D funds. I^hirs included performance of a cost analysis 
for each project, with a focus on the r^plation of cost to student outcomes and project 
process measures, and an assessment jof the influence of projects or their compo- 
nents on other schools and school systems in their states. In addition, a descrip- 
tive study and analysis of projects supported by state administered Pai*t D funds, 
paying particular attention to different state allocation strategies, was to be 
performed, ^ 

The basic rationale for the 5tudy was that an evaluation of the first three years of 
the rjart D program would lead to improved implementation of the Part D program 
during subsequent years and would cohtribute to the replication of successful pro- 
gram aspects by local school districts. Since the Part D effort was so closely ' 
associat^ci with the concept of career education, it was also expected that informa- 
tion might be obtained in the course ofTh^ study which would assist in operational- 
izing the concept of career education at the local level. 

Principal Objectives: 

^'he basil^.th-rust of the study was to obtain information which would be of assistance 
in the continued operation of Part D and related programs, as it was recognized 
from the outset that an evaluation conducted at the end of the three year-cycle 
could not address the program's long term goals. Accordingly, the objectives of 
the study were defined a^ follows: '. ' . t ^ 

1. Evaluation of the effectiveness of federal discretionary (Part D) projects and 
their components where effectiveness is defined as the extent to which student 
outcomes attributable to project activities confornl to the legislative intent 
for Part.D. , 
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2. Performance of a cost analysis of eaph federal discretionary project cover 
the three-year per iod of PartJLi*«taing with focus on th^ relation of cost to 
student outcobies and project process measures; and performance of cost 
analysis for those discretionary projects whose funding has terminated, 

^cx)mparing the period before termination of Part Q^fun,ding to the period 
^a'fter its termination. ^ * 

3. Assessment of the influence of projects or their components on other sd^iools 
and school systems in^eir states • ' * 

4\ Perforn>^ce of a descriptive' study and Analysis of^projects supported from 

state-administered fund?, pacing particular atten^on to their estimated impact; 
and identification of different state allocation strategies^ to determine the 
differential impact 6f var^us strategies. 

Contractor: - i ' ' 

The stuJiy was initiated and completed under contract #HEW-OEC-0-73-6663 by 
Development Associates, Incorporated, Washington, D. C. The effective starting 
date^of the contract wa's June 19A 1973, and the final report was submitted in ^ 
March 1975. Th'e field work was performed from February through May 1974. 



11. METHODOLOGY 

The study approach incltKied review of available documents and onsite visits to 50 
projects and State Departments of Education. During the first three-year cycle of 
Part D funding,^ 61 federally administered projects were supported in the 50 states 
and District of Columbia. The 50 projects included in this study were selected to 
meet the following criteria: First, one project in each state with a first- rovind * 
project operating in FX '73 was to be visited; as Hawaii had nip^uch project, 49 
states and the District of Columbia wer^^isited. Second, each project visited 
must have proposed to focus activity at the elementary school, junior high school, 
and senior high school levels. Third, if more than one project met the second 
criterion, the first project funded was selected. 

Within each of the 50 projects selected,' the broad design called for administering 
questionnaires and tes^s to both participating and npn-pa^cipating stu^dents in 
-gr^tdes 6, 9, 'and four strata in grade 12. These grades were chosen because they 
represented the terminal years for the differing curricula and objectives generally 
assigned at the elementary, junior high, and senior high levels; they would thus be 
expected td be the grades where) students had received the maximum impact. 

In addition, a random sample of participa,king teachers and counselors as well as " 
non-participating 6th grade teachers wais purveyed. Also, inter^sviews with project 
directors and other staff were conducted and project records.-and^reports were 
reviewed. 

At the. state level, the state director of vocational education or his desig^^e.and 
the dirrector of the state Research Coordinating Unit were interviewed wi&j.^espect 
to the federally administered and the state administered Part D projects^ /ill data 
collection instruments were cleared for use by OMB and assigned No. 
A detailed field manual was prepared to assist in the data collection effort and iield 
staff received intensive training prior to beginning the field effort. 

Although the number varied according to the size and nature of the projects, a 
minimum of Sjo of the students and faculty were surveyed in each of the 50 projects. 
In total, 4, 632 participating and 4, 043 non-participating students were tested; 
the number of participating teachers siirveyed was 1,43 3, and the number of 
participating counselors was 229. The 'data gathered were processed and synthe- 
sized using standard statistical techniques. Analysis of student data was performed 
for each sampling group both on a project-by-project basis and across all projects. 

^ in. FINDINGS AND Co'nCLUSIONS 

.J 

Prior to summarizing the findings and drawing such conclusions as they permit, 
the positive program management actions which USQE has taken on the basis of 
this s.tudy's preliminary findings and other reviews of the experience should be 
noted. Specifically, concerted efforts have been made to be more specific with . 
respect to the definition of key terms, and the student level outcomes expected. 
In addition, a major emphasis 4ias been placed on improving the quality and prac- 
tical utility'^^of individual project evaluaticms. These. USOE actions address the 
overall study finding that projects were ^typically not well defined in terms of pur- 
pose o^clientele and that this lack of clarity may relate to the failure ^in many^ 
projects to identify student outcomes significantly related to project activities. 
Thus, the efforts at the national level during the last yean to be irio-re specific 
with respect to both program objectives and. managerial practices regj'esent explicit 
attempts to improve the program In^he^ears ahead. - - * 



A. Summary of Findingsr ^ , 

The intent of the Part D program was to fund projects which would implement 
a c titties .-expected to assist sjxidents in obtaining satisfying employment. In ^ 
'essence, the grant awards may be considered as the planned federal fiscal 
inputs into each project, the local budgets as the projects' planned inputs, and 
expenditures as the actual inputs, A review of the year-by-y^ar input data 
revealed that in most projects in no single year did the inputs occur at the level 
planned. Taking the three years in the aggregate and considering bothVexpenditures 
and the legislative expectation that funds would be expended over a thre.e-year 
time period, it was found that in 37 out of the 50 proj^ts-visited (74%) the federal 
dollars expended were below the level planned. 

In part, the^enerally high leVel of project underexpendituie s was attr.ibuted to the 
timing of the federal grant actions and the first year start-up* requirements in many 
projects. Another partial explanation for the* underexpenditures may be related to 
the nature of the expenditure records maintained by project managers. Many , 
project directors did not have expenditure data which permitted them to assess 
project status in even gross fiscal terms. In addition, in the great majority of , 
cases, they did not have information which related expenditure? to any set of projecr 
activities. 

From the findings it was apparent that onflan annual basis, and across the three years- 
of program operations, the extent to which projects carried out the activities speci- 
fied in the USOE policy paper, * which governed the federally administered projects^ 
varied considerably across the 50 projects. While most projects reported students 
in most of the USOE required activities at some point during the three years, 
relatively few (26%) reported having students in all activities. \ ^ \ 

The findings with respect to project activities may be explained partially by the data 
repO]*ted pertaining to program management. A review of the stated objectives of 
the 50 projects revealed that in many capes the activities called for by the policy 
paper were not addressed. In addition,' a comparison between stated objectives of 
t^rojects and activity i: a tegories indicated that in many projects thfe performance of 
activities could not be related to the stated objectives. 

Based on- analyses of student responses across all projects, the greatest impact of 
the Part D program was at the elementary school level. In general, tl^e program 
appeared to have had less^impact on students at the 9th grade level; at' the 12th grade 
level, impact was less than at the 9th for two sampling groups and greater for the 
other two. * 

On a proJect-by-project basis the impact of the program on student* was small, 
with the bulk of favorable outcomes in each student group ^onfined to a small grpup 
of projects. These "favorable" projects were somewhat different for each student 
group. Out of 45 projects where comparisons were made between participating §ind 
non-participating 6th graders on six relevant* student outcome indicators, there was 
a difference in favor of participants for three or more indicators in ten projects; 
thfe most outstanding-project at^this level reported differences in favor of partici- ' 
pants on five of the six indicators. Of the 42 projects where comparisons wefre 
possible at the 9th grade level, one project scored positively on six of the nine rele- 
vant indicators, and ten produced positive results with respect to three or more 
indicators. At the 12th grade level, nine indicators of student outcomes were 
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relevant foF each of the four stiatim sampled. Students whose participation in thb 
program consisted of having teachers who utilized concepts and/or materials made 
available through the local projects in their regular teaching program (i.e., infusion o 
career education concepts into the scJiool cur ricttlum), demonstrated a significant 
difference from non-participants on six indicators in one project; in five projects 
such differences were found on two or mor'e indicators. In none of the projects 
where comparisons were possible for students in the, Work Experience, Skill Train- 
ing, or Guidance and Counseling groups was a significant difference between parti- 
cipants and non-participants found on more than three indicators. 

The search for relationships between the outcomes and selected project treatment 
in<^icators did not provide a clear set of relationships bet\^een actjivity or treatment 
indicators and student outcomes. However, the rqview of responses of non- 
participating sixth grade teachers suggested that ip at least some projects the 
inability to measure significant differences between. participating and non-partici- 
pating students may relate to non-participating teacners implementing career 
education concepts in their classrooms. 

Anqther potential explanation is suggested by the relatively clear ::elationship which 
was^found between project expenditures and student^ outcomes . With respect to 
elementary, junior high, and senior high familiarization activities, positive indica- 
tions of project effects on students occurred where relatively more project funds 
were expended on relatively fewer students. 

In summary, projects typically were found to have addressfed four components with 
respect to, the federal activity areas. Virtually all projects had, some level of 
fami^arization activities at both the elementary and junior High school levels. 
Projects generally devoted 26% of thef^^art p funds to- pay for elementary school 
activities which involved approximately 2; 000 students per project. The students 
participating in elementary activities usually amounted to less than half (46%) of 
the total elementary school enrollment of thfe grantee and accounted for over half 
of the total number of project participants* (52%)., 

Junior high school familiarization activities involved some 47% of the total junior 
high enrollment of the grantees and averaged 1,400 students per project* Th^ 
activities at this levpl involved approximately 34% of the total number of project 
participants and cost 29% of the Part D funds. ^ ' \ 

Senior high school activities involved some 650 particip2^t§ per project and repre- 
sente'd approximately 35% of the total high school enrollment ctf the grantee. High 
school participants represented approximately 15% of the total number of Part D 
project participants and high school activities accounted for 44% of the total project 
cost^. At the high school level, the typical project reported participants in famil- ^ 
iarization activities and one other activity, either work experience or job entry 
skill training. p 

The USO& policy paper governing the first round projects required that grantees 
make provision for the continuation of project supported activities from "regular" 
funding sources after Part D support had terminated. In 14 of the projects visited, 
school personnel indicated that projedt activities either had or would terminate at 
the end of the grant period, and in 19 other cases they indicated that activities would 
be reduced. In eight of the projects, project activities either were contiivuing after 
the termination of the grant or definite plans existed for the continuation*^' In nine 
cases,., activities had actually expanded after Part D fundir/g had ceased. 



B: Overall Conclusions 



Well over half of the teachers and counselors surveyed- iij each of the 50 sites 
visited indicated that in their judgment it was importaint to include career education 
in the school curriculum.- From this point, ot view, the first years of the program 
may be judged to have had a substantial effect. However,' in general, neither the 
federallyNsponsored activj.ties nor the federally expected student level outcomes 
of the prdgram cyecul-red at the level planned. While a number of reasons for this 
are possible, the findings suggest that the most likely are associated with the 
general lack of a set of clearly defined objectives, definitions, and managerial 
requirements and procedures at both the project level and at the federal level. 
More specifically: 

• The definition of key terms and concepts was neither precise nor con- 
sistent at either the federal or local levels. For example, students were 
identified as project participants because they were being taught b^ parti- 
cipating teachers, but what constituted teacher participation varied from 
attendance at a two-hour career education workshop to ten or more 
released days per semester for inservice training, curriculum develop- 
fnent, and classroom planning. This failure to establish. operational 
definitions and categories contributed to the ipability of projects to identify 
with assurance participants in the programs and to the inability at the 
federal level to monitor project efforts effectively. 

• Budgets and expenditure records typically were based on "line-item" 
rather than programmatic ac^tivity categories. Determinatiqn of activity 
costs was very difficult- This difficulty was primaril;^ a result of the 
grant application and award process which-did not specify costs by 
activity; only in the aggregate. This was further complicated by no provi- 
sion for administrative posts, which meant that most projects attempted to 
prorate such costs and attribute them to treatment activities. It is probable 
that this contributed to the failure of project directors to analyze expendi- 
ture data and of project staff to use budget and exjjenditure data,, in the man- 
agement of projects. In addition, ifis probable tha,t-this contributed both 

to underexpenditures and the failure ^to engag'e in expected sictivities. 

• Similarly, USOE did not use fiscal data as management indicators. ' ' 
Typically, the fed^al management staff did not receive or analyze expen- 
diture data. This contributed to the underexpenditure of funds annually 
and the support of some projects for perio<f8 in excess of^the three-year 

J funding limit stipulated in the legislation. ^' 

• The evidence strongly indicates that exemplary programs require consid- 
erable start-up activity and time. Failure to anticipate this adequately 
appears to have resulted in the inability of projects to meet program 
expectations. 

• Generally, participaftts in the projects were e3q;>08ed more to visitors in 
their classrooms who discussed careers, and went on more field trips to 
learn about jobs; than non^participants. The data suggest that this quantl-;^ 
tative difference in the ni:iixiber of such experience's was not sufficient to 
produce a measurable impa^ct on students. Rather, dt appears that such 
activities need to be Integrated Into a well-planned and comprehensive 
effort. ' , • s ' 
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The number of different approaches to building a bridge between school 
and earning a living undertaken by the first round projects was limited. 
Fewer than half of th^ projects had work experience or skill training 
activities. Many of the new approaches suggested in the legislation and 
Federal Register, such as exchanges of personnel between schools and 
agencies or businesses, were not attempted during the ^irst round. 
Projects generally did little to promote cooperation between public 
education and manpower agencies. ^ 

The prinaary focus o£ round one was elementary and secondary familiari- 
zation and orientation. Most of the total funds and m^t of the student 
participants were engaged in such activities. Work e^^rience and skill 
training efforts tended to be expensive and involve limned numbers of 
students. Where other activities were initiated, such as alternative high 
schools, they tended to be less integrated into the school system, expen- 
sive, and by design, served fewer students (i.e., potential dropouts or 
others with special needs). , • > 

To assist college and non-college bound students in obtaining employment, 
the USOE policy paper indicated that projects should provide specific train 
ing in job entry skills to students not previously enrolled in vocational 
programs ju|t prior to the time that ^ey leave school. The data appear 
to indicate that most projects extended such training only to the non- 
college bound. — 

The projects typically did not assist students* in securing employment, 
either during school or upon grac^uation. Generally, the placement 
activities implemented were essentially referral serviced and little or no 
foUowup was provided. In general, neither project staff nor school 
.personnel maintained records, of referrals or placement activities. 

Tlfe student responses* indicate that guidance and counseling at the high 
school level was viewed by students as helpful to the]Jn. The data also 
fhdicate that a great mamy st^idents did not feel they had sufficient 
Opportunity to receive assistance from their counselors and they would 
have lU^ed to have increased their contacts. 




Tbe Federal Register indicated that applications for round one projects would be 
judged in part upon the projects ' objectives being "sharply defined, Qplearly stated, 
capable of being attained by the proposed procedures, and capable bf being measure 
During the first years of Par6 D, neither at the federal level nor in most school 
districts was this the case. .The^nability to define clearly federal program objec- 
tives undoubtedly contributed to many of the difficulties discussed in this study. 
Without a specified ^et of federal pxpectafcions 4h^ individual third party evaluations 
'could only relate to local project^^als and objectives. This resulted in a large 
diversity of evaluation approaches. It is not surprising, therefoite, that the^e 
evaluations did not provide meaningful or useful management information at the 
federal level for comparing projects. Without specific criteria for judging^?! 
project progress, t}>e USOE really had no basis for making annual or mid-project 
grant modifications. Most importantly, without such predetermined criteria, the 
USOE had little indication of what criteria would be employed in the final evaluation 
and so were*not able to "manage effectively foiv success. 
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A final point in this Vega'rd relates to the whole area of the management of 
e<%cational projects. While this was not a management practices study, many of 
the findings and conclusions appear to relate directly to management issues/ The 
points raised above pertaining to advanced'planning, the clarity -of program and 
pnoject objectives, and the use of fiscal data are management questions which 
relate to attainment of student level outcomes. So too, we suspect, are the^oints 
made in the text pertaining to the inappropriateness of the job descriptions of - 
project staff, failure to maintain records pertaining to placement^and other 
project activities, and the limited focus and use of third party evaluations. 
Based on our observations, project staff typically were sincere, hard working, 
and oriented toward the substantive aspects of the program.^ T^ically, however, 
they were not well versed in the use of sound management practices as an aid in 
bringing about desired innovations in their school districts. Thus^, based on the - 
experience of the first round projects studied, it appears thett a USOEiocus on \ 
providing guidance and assistance in the area of project management would hav^e* 
been of majof benefit to the successful implementation of exemplary programs. 

Such conclusions as these are not uncommon in nation als tudies. ^hile they are* 
accurate, they ar^ also somewhat misleading, for they cannot and do not cover /t, 
pre -prog ram context of the system* of federal funding. Legislative mtent is 
frequently less than clear, the time constraints d6 not usually perntiKcareful . 
planning at the federal level prior to ^funding of local projects; the federal agencies 
are frequently not sufficiently staffed to permit effective management and, when 
they are, the desirability of effective federal management of, local educational 
programs becomes a question, . Thus, as long as inherent weaknesses such as 
these persist, evaluation findings will essentially tend to be ei thfr negative or 
ambiguous and recommendations will be symptomatic instead of definitive^ 

In conclusion, the USOE and especially the program sfaff in DVTE, w^o have from • 
the outset not only Cooperated wit*]^ this study but also have taken actioyis based on * 
preliminary findings, should,,be cpnunended. It is rare for a national program with 
innovative and ambitious objectives to engage in comprehensive, ,imp^ct-oriented 
evaluation after only three years of operation. While sortie of 'the firfdijigs of the 
study me^y not be what one might have wishe^d, .given the complexities ijiltgrent in'^' 
implementing the Part D program during its initial years, they ought fipo to be 
surprising. In our judgment, perhaps the most srgnificant s^spec^^of the studJiTid 
the inference which may be drawn from the fact that it wa^Jpone, and' that a/Aions 
have been taken^as a result. Clgarly, USOE, and the staff of DVTE in psjrticular, 
have evidenced a commitment to improving both the conte^n^ and the managem^t 
of federal education programs.' ^ 

r ' ' . ■ ■ ' . 
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CHAPTER I: INTRODUCTION 



Part D of the 1^68 amondments to the Vocational Education Act of '1963 provided 
for the funding of exemplary projects at elementary and secondary ^choo4 levels 
in each state and territory. As specified in the legislation, half of the funds for 
the Part D program v^ere to be administered directly by the federal government 
and half to be administered by the states and territories. At the end of the first 
three year funding cycle of the program, it was determined that an evaluative 
study of the first years of program operations should be conducted. This report 
presents the findings and conclusions of that study. 



As outlined by; the U.S. Office of Education, the first issue to be addressed injj2 £_ ^ '_ 
evaluation yc^s the effectiveness of the f e d ^ r aJ[Jx-Spon^so-r ed^ParTTT^rp rojects. 
Effectiveness was to be m^^.^ULV-e^^y'^^TKe degree to which s^dent outcomes 
attribata43le to project activities conform tq the legislative int^t for the Part D 
funds. " By virtue of this critefion for ^sse^;sing program effectiveness, consid*- 
erable empha-si-s^a the chapters which 'follow is placed on the bacWround of the 
Part D program and the conceptual framework which governed both the design'of 
the study and this re*port. ' , ^ 

• Throughout the process.of designing and implementing this assessment of the (i^st 
three years of the Part D program, close attention was paid to the legislation and 
to the issue of attribution of outcomes to Part D efforts.' At the outset, ^an agree- 
ment was made between the study team and the relevant offices within the U.S. 
Office of Education that activities undertaken at the local level which could not be 
related to legislatively based categories of program'activity would be excluded 
from consideration. It was also concluded that even though th$ OSOE policy 
guidance given to Part I|^projects "* did^ not address certain of the legislative^^ 
activities, data with regard to all of the legislative activities and their associated 
results would be obtained. Finally, it was agreed that since the clear intent of 
Congress wasr to provide suppcfrt for activities which would produce student out- 
comes, every effort would be made to assess th^e level of student impact of pro-ject 
activities, although effort was also directed at teachers and other school personnel. 

These crucial decisions ma^e at the start of the study were basic to the overall 
study design and hence to the findings w^ich *ai e presented in this report. As a * 
basis for interpreting the findings which follow, it should be understood clearly 
at the outset that each of these decisions not only contributed significantly to t\ie ' 
nature of the findings of the study but also were somewjiat bold.- It was clearly 
understood by the line administpa.tors of the Part D program that this design coupled 
with what they knew, qr suspected, about the exemplary projects at the local level 
would produce, results which some would view as quite critical of the overall effort. 
^Equally clear* from the outset, however, was the even more basic decision on the 
part of the program administrators that they desired as clear and accurate an 
asJ^ssment of the ^irst round effortsoas possible.* 

It should be noted at this pointythat the Part D program was^ established to permit 
the implementation of concepts and approaches which, while supported by theory 
and research, were on the frontiers of vocational education. As implemented under 
the direction of the small program staff of USOE's Division of Vocational and 
Technical Education, the fedferally administered projects focused on concepts which 
over the next several years came to be associated with the term "Career Education.' 
The first set of Part^ projects, which are the subject of this study, are considered 
b>s.many to be the, earliest federally supported efforts to implement on a compre- ;^ 
hensive basis career education programs in local school distriet^. With the 

■ , - ■ , ^ f 

* USOE Polic> Paper AVLV70-1. October 2 1969 \'diffcrcnt policy paper. AVfE-V72-10, August 28. 1972. has 
guided the operation of projects funded since lh*it date * i ' , 



termination of these initial projects, a. second group of projects was funded under 
a ni^dif led'set of program guidelines which'miyre esq^licitly iocused on what by^^en 
were considered Icare^r education objectives. As this study began, a third set of 
projects with a fdcus Vj-hich reflects the insights gained from the easier .efforts ^lus 
the increasingly ^ide acceptance of carreer education were being planned. Thus, at 
the time this study was being planned, it was apparent that the program would con- ' 
tinue, that the basic concept^ associated with Part D and career education in g^eneral 
w ere most favorably received by. a wide audiepce, and that in the long ru\i much was 
"to be gained by an assessment of the first years of the exemplary efforts na tej-ms 
that would cont?ribute to improved program and policy decisions in the future 

It was within this general context that the study was designed and implernented'. 'As 
a result, the purpose of this report is^not primarily to presenba detailed picture of 
the achievements of*a set of projects which have now terminated, which by statutory 
hmitatiorf are inelig^ible for further federal funding, ^and who6e sister projects in 'Ihe- 

second round" of Part D funding are g-uided by a different USOE policy paper. Not 
.s the primary intent to offer definitive conclusions with respect to the manner in 
.whrch this particiilaj educational program was administered during its most forma- 
tive years. While tc^ some extent this report does both of these, its more im.pOrtan^ 
purpose is to assisMn the further development and implementation of ^art D and 
relatied' educational- ejfforts . ^ ' ' 

Fmairy. before pro<^*eding to the body of the report it must be stresses! tjiat this 
study took place after the firs-t round programs had been in operation fqTr^at^least 
three yeaj^s. In nriany cashes, formal funding had 'ceased and key perso'nnjer"were no 
longer in the schoSl ^^istricts at the time of the visits. In addition, as they^were 
charged to dj, many|of the projects had been highly successful in stimulating the 
implementation of^^'reer education concepts in other school systems. Indeed, from 
our'visits to 50 projects and our contact with federal staff, i^.was I'eadily apparent* 
.that all associatedSvl^h the program werte sincerely dedicated and. hard working. 
Nevertheless, ^fie combined effect of the at)sence of key personnel, the ab^se^ice of^ 
baseline information .with respect both to the student ot^c om e^ Indicators selected ^ 
and the activities in force in schools visited when they first iJ'eceived Part D funds, 
and finally, of pkrojec^ success in dissemination of materials and- approaches is 
undoubtedly reflected in the absence of differences between participants and non- 
participants in some of the study outcome findings. The findings and conclusions 
presented in the chapters which follow should point toward pr ocedur€fs%,and policies 
which will lmpro^^" ti^e implementation of Part D and career education efforts in the 
future. In^hiS negaud, it should be noted that appropriate USOE offices have already 
taken significant at^fons to overcome these problems. In a very real senSe , ' then , 
this report should pe viewed as part of an ongoing process to d^velop ckreer educa- 
tion in the nation's? schools. " , ^ 



CHAPTER II: BACKGROUND 



In June, of 1^73, the first of the fedcreil discretionary Part D projects were 
terminating their third and final year of operation. These projects constituted the 
earliest efforts funded under Part D, Section 142(c) of the Vocational Education 
Act of I9h3, as amended, and Aere part of the first round of Part D funding. The 
proiects were among the initial federally supported efforts in career education and 
have been considered as in the vanguard of the career education movement. The , 
purpose oC this section is to discuss briefly the series of events which led to their 
funding, the high points of their first years of operation, and to*sketch what 
transpired from the national perspective the following yeac ^ith respect to this 
study. It provides the historical background and objectives for the detailed 
analysis of these projects which is the subject of this report, 

A, Part D: Historical Overview 

Part D of the Vocational Education Act was a direct result of recommendations 
from a special Advisory Council on Vocational Education appointed by the Secretary 
of Health, Education, 'and Welfare in Noverhb'er I96(). The Council w^s chaired 
by Martin Essex, state superi^fitendent'of public instruction in Ohio, and included 
members from business, labor, local school districts, universities, and private 
foundations concerned with vocational education and manpower development, 

.Paying particular attention to innovations and new directions emerging from ^ 
research and development, the Council conducted a detailed study of the status of 
vocational education in the United States. The Council completed its work in 
December 1967, and published its General Report containing legislative and 
administrative 'recommendations the following year,' The Council's recommenda- 
tions w^ere based on five operational pr inciple^^for vocational education: 

1, Vocational education cannot be meanj-Qefully limited to the skills necessary 
for ^ particular occupat.ion. It is mo rejvappropriately defined as all of those 
aspects of educational experience which help a person to discover his talents, 
to relate them to the world of work, to choose an occupation, and to refine. 

^ his talents and use them successfully in employment. In fact, orientation 

and assistance in vocational choice may often be more valid determinants of 
employment success, and therefore more, profitjable use of educational funds, 
than specific skills training, 

2, , , , , Where complex instructions and sophisticated decisions mark the 
boundary between the realm^of man and the role of the machine, there is no 
longer room for any dicHotomy between intellectual competence and 
manipulative skills and, therefore, between academic and vocational 
education, 

3 , ' , , , , Education cannot shed its responsibilities to the student land to society 

in his behalf) just because he has chosen to reject the system or because it 
has handed him a diploma. In a world where the distance between the 
experiences of childhood, adolescence, and adulthood and betwe*en- schooj 
^nd work continually widens, thp school must reach forward to assist the ^ 
student across the gaps just .as labor market institutions must reach back to 
assist in the transition, , , , - 

4, Some type of formal occupational pre^jaration must be a part of every 
educational experietice, , , , In addition, given the rapidity of change apd the 
competition from generajily rising ^^icational attainment, upgrading and 
remedial education opportunities are a continual, necessity. 



5/ The objective of vocational education should be the development of the 

individual, not the^ neeHs of the labor market, , . , The system for occupatibnal 
^ preparation should supply a salable skill at any terminal point chosen by the^ 
individual, yet no doDrs should *be closed to future progress and development,-^ 

Building on these principles, the Council reocJ^nmended a "unified system of 
vocational education" which included the following as key components: 

1, Occupational preparation should begin in the elementary schools with a 

realistic picture of /the world of work. Its fundarnental purposes should be 
to familiarize the student with his world and to provide him the intellectual ^ 

^ tools and rational habits of thought to play a satisfying role in it. f 

o\ ' \ " * 

Z. „ In junior high school, ectmcrfnic' oriehtatio^J and occupational preparatio"^ 

. should reach a more sopfhisticated stage Awith study by all students of the 

/^economic and industrial system by which- goods and services are produced 

and distributed. The objectives should be exposure to the full range^of- 

\y occupational choices which will be available at a later point and full } 

knowledge of the relative advantages and the requirements of each. 

3. ' Occupational preparation should become more specific in the high school, 

though .preparation should not be limited to a specific occupation. Given 
the unce^itainties of a changing economy and the limited experiences upon 
which Vocational choices must be made, instruction should not be overly 
• ' narrow but should be built around significant families of occupations op^ 

industries which promise expanding opportunities. . All students outside 
the college preparatory cu'rriculurn should acquir^ an entry-level job Skill, 
but they should also be prepared for post-high school vocational and technical 
education, ^ven those in the college preparatory^ curriculum might profit 
from the techniques of learning by doing. On the other hand, care shoi^ld be 
taHen that pur)suit of a vocationally oriented curriculum in the'high school 
does not block the upward progress of the competent studei^t w^dr&later ^decides 
to^pursue a college degree. ' ' 

4. Occupational* education should be based on a spiral curriculum wliich treats 
concepts at higher and higher levels of complexity as the student moves 
through the program. *Vocajtional preparation should be used to make general 
education concrete and understano^ble; general educal/ion should point up the 
vocational implications of all education. Curriculum materials should be 

^ prepared for both general and vocational education to emphasize these 
relationships. 

K ' ^' ' 

5. Eeyond initial prepat^^tion for employment, many, out of choice or Necessity, 
will want to bolster an upward occupational climb with part-timei and 
Sometimes full-time, courses and, programs as adults.^ These should be 
available as part of the regular public school system. They should npt be 
limited to a few high-demand and^low-cost trades, but should provide a range 
or o(ccupational choice as wide as ^^ose available to students preparing for 
initial entry. * - 

6. Occupational preparation need not and siiould not be limited to the classroom, 
^to the school shop, or to the laboratory, ^any arguments favor training on 

the job« Expensive equipment ^need not be duplicated. Familiarization with 
the environment and discipline of the workplace are important parts of 
occupational preparation yet are hard to simulate in a classroom. ? 
Supervisors and other employees can double as instructors. And^he trainee 
learns by earning. On the cJther hand, the employer and his supervisors 
may be mor^e production than training oriented. The operations and 



'/ ' ' 

equipment of a particular employer may cov?r only part of a needed range 
•of skills', necessitating ttfensfer among employers for adequate training. Jhe 
ideal is to meld the advantages of institutional and on-the-job training in 
formal cooperative work-^stuVly pro grains. 

7. Effective occupational preparation is impossible if the school feels that its 
obligation ends when the student graduate^. The school^ therefore, Vnust 
work with employers to build a bridge between school and work. *w Placing tlie 
student on a job and following up his sifccesses and failures prov^e^the best 
possible information to the school on its own strengths and weaknesse^.^^ 

More directly relevant to Part D, the CounciV-s third legislative recommendation 
was that "funds and permanent authority be provided for the Cornmiss io'ner of 
Education to make grants and contjj^cts to state boards and with the approval of 
the state board to local educational agenci^syand to oth-er public or ntjnprofit 
private agencies, organizations, or i;istitutjoTis Jor planning, ^development, and 
operation of exemplary and innovative progframs of occupational preparation. " 

The Council noted that J'the effectiveness of a Viumber of new methods, techniques, 
anci services has been verified in research and experimental studies" and that it 
is necessary to incorporate these new methods and services as they become 
available "in order tc^^ensure that *all youth and adults, especially those with 
disadvantages, have adequate and appropriate opportunities to prepare for 
satisfactory employment, , , . The Council then specified that "these exemplary 
programs of occupational education should include the following provisions which 
a^re not offered widely in existing vocational education programs: 
. ^ ?, • 

Exploratory occupational education to provide practical and «• * 

educational experiences essential to understanding the demands ' ^ 

and complexities of our modern society and opportunities in - j \ 

the constantly changing world of work; ^ 



Programs designed to acquaint 'students with employment 
opportunities an^-fp^teaich skill and knowledge required in 
^ one or moi^e indusfries or families of dcdupations 'certified 
by thfe U,S* Department of Labor as offering expanding 
opportunities for .employment; 

Programs or projects to provide students with-'ve<iucational 
experience. through part-time work which wiUjfcssiat in their 
^ maximum development and which will help link school and 
employment; 
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Guidance and counseling to assure that all students' interests > 
and capabilities are developed, in relation to their caree'r 
objectives and to ease the transition from school to work by 
assisting them in initial job placement; 

'Improvement of curricula to stimulate broad-scale innqyative 
changes to provide more realistic vocational e^ducation programs 
for youth and acfults at all sk5ll levels*^/ V*^ 

V . ' ' ' 

In essence, the Advisory Council's 'Legislative Recommendation No. ^3 was the 
basis for Part D of the Vocational Education Amendments of l^A« The 
e?tpressed gcJal of Part D is "to reduce the contii^uing seriously high level of 
youth unemployment by developing mea^s fok- giving the same attention as is now 
given to the college preparation needs of those young. persons .who go to college, 
to the job prepartition needs of the two out of th^ee young persons who end their 
education at or before completion of the secondary level, ♦ . , " Based on this goal, 

•• - - - 5 - ^ \ • . 



the legislation states: "The purposes of this 'part, therefore, are jto -Create a 
bridge 'between school and eai<ning a living for young peopfe, who are still in 
school, who have left school eif^teissby graduation or by dropping out, or who are 
in postsecondary programs of vocational preparation, an4 to promote cooperation 
between public education and manpower agencies. " 21/ ' ^ 

In order to build the bridge between school and earning a living and to promote 
cooperation between public education and manpower agencies, the legislation 
provided fund^ to be used to "carry out the development, establishment, and 
operation of exemplary and innovative occupational education programs or projects, 
designed to serve as models for use in vocational educatiQji prograrti^sl^^ The 
funds afloc$ited to 'Part D were to be distributed am&ng the^fifty states and the 
District' of Columbia according to a formula specified in the Act with up to 3% pf 



the total allocated to be, distributed among Puerto Rico, the Virgin Islands, Guam, 
American Samoa, and t/he Trust Territories of the Pacific Isla"nds according to 
their respective needs^for assistance. Of the amount available to each state, 30% 
was to be administered by the state and 50% was' to be a'dministered by the U , S\ 
Conimissioner of Education. * » . ' 

For both the state ai^d the federally administered portions of Part D, the legislation 
specifies that grants' 6r contracts may be made to pay all or part of the cost of 
planning, establishing, operating, or evaluating exemplary programs or projects 
designed to carry oAit the two main purposes th-e Act and "to broaden occupational 
aspirations and opoorttlnities for youths, with special emphasis given^to youths who 
have academic, socioeconomic, or other handicaps, w^^h programs of pnrojects 
may among other^, include .4 ^ 

(A) thosG^ designed to farj^liarize elementary \nd secondary school / 
stud^ents with the broad range of occupations for which special 
skills^are required and the requisites /or careers ii^ such 
occupations; 



(B) programs or projects for students providing educational 
e:jcperience& through work during the school year or in the ^ 

'imer; ' . , 

(C) programs or projects. for intensive occup^ational guidance ^ 
' . and Counseling during the last years of school and for initial / 

job placement; ' 

programs or projects designed to broaden or improve 
vocational education curricula; 

exchanges pf personnel between schools and other agencies, 
• institutions, or organisations participating in activities to 
achieve the purposes of this part, including manpower 
agencies AndJndustry; ^ ' ' 

|3^rograms or j^rojects for yd^ng workers r*eleased from * 
their jobs on a part-time basis for the purpose of t 
increasing theii; educational attainment; and, 

. ' ' \' ■ 

programs or projects at the secondary level to motivate ^ 
and provide pr eprofessional preparation for potential 
teachers for vocational, education. " 

With/the passage of the legislation in October 1968, operational respojjsibility for 
PartlD'^as given to the Division of Vocational and Technical Education in the U.S. 
Office*^of Education's Bureau of Adult, Vocational, and^ Library Programs. In 

9.7 ^ 



preparation for implementing Part D and related amendments, USOE sponsored 
a conference in Atlanta in March 1969, at which a pum^ber of papers pertaining to 
implementing aspects of the legislative program were presented. Among those 
presented were: "Unifying an Entire System of EducatiV)n Around a Career 
, Development Th^me, '''^by EdNJwn Herr, and "Elements^of a Model for Promoting 
Career Development in Eleme^itary and Junior High Schools, " by Norman Gysbers.i-^'' ' 
The titles of thes^ papers suggest the direct link which exists between the origins 
of th'e Part D effort and what several yc-'ars later came to be called "career 
education. " 

In Octobejp 1,969, the U.S. Office of Education invited interested school districts 
and other organizations througho^jt the country to submit proposals to conduct Part 
D*projects. More than 175 proposals were submitted to U50E in response to this 
invitation.—'' Base^ on^review conducted in accordance with provisions cited in tho 
Federal Registe r— USOE ]selecti?d one project in each, state to receive Part D 
funds >for a period of three years. The. review process included provision for 
states to comment on and veto proposals from organizations in their states. The 
review was sufficiently stringent that in some states all of the initially submitted 
proposals were rejected and allocation of funds postponed until a proposal meeting 
USOE^criteria was received. As a result of the review proce'dures as well as 
variations in the time taken to respond to the invitation lor proposals, the initial 
Part D projects did not all start at the same time. 

The legislation provides that a project is noTto^eceive Part D funds for more than * 
three years and tlie USOE invitation for proposals specified that projects fvould be 
three years in duration. Since it was anticipated from the start that 'ont-e the first 
project in a state was terminated another would ^e funded, She initial projects are 
, commonly referred to as those funded in 'the "First Round." In Jun^ of 1973, the 
first of the Second Round projects were funded by USOE utilizing criteria and 
insights gained during the first three.years of operation, and by August of 1974, 47 
Round II projects had been funded. 

In total, 6h projects were funded by USOE in the first roimd m each of the 50 
states^ the District of Columbia, and the territories. For various site-specific 
reasons', three otthe ea'rliest funded projects were terminated significantly 
prior to.their expeoted three year diira'tion. In two of these cases\ another 
project in the state was awarded funds for a three-yeaivef^ort and was considered 
a part of the fir st- round effortl^^In seven cases it was determined tbat the recipient 
of the initial grant would not need all of the funds availa^ble through the legislative 
allocation formula and subsequently a second, and in the case of California a thi;-d, / 
project was funded ajnd considered part of the "first round." Table 1 on the following 
page presents the starting J|ates ^for the fir6t funded project in each state in the first _ 
polumn and' in the second, column the starting date for subsequent projects which are 
considered as f^rst'round un.dertakings by virtue of receivffig funds prior to January 
1973. • \. 

- > 

As will become cl-ear in subsequent chapters of this report, there was considerable , 
diversity: among the first round projects in terms of administrative structure, 
specific project objectives, and operating style. USQE awarded grants in sorne 
cases to state boards of education to encompass several school districts and in 
'"other cases, to single school districts to focus on a specific portion of the school 
district. In every case,- however, f^e grants were made in response to proposals 
judged consistent with a policy paper (AVL V70-1) which constituted a portion of / 
the invitation for proposals. 



Hawaii. Minnesota, and Florida. 



\ 



- W 



TAHLE 1 ^ 
FIP<;T ROUN'D C0MMISS[0NER'S share part Dfc) PROJECTS SY MONTH or ORIOIWAL FUNDING 



Month. 



First 

Funded 

Project* 



June * July Au^. Sept Oct. Nov D« 



Ala 

Ark. 
FU. 
Ca. 

Hawaii 

Ken. 

La. 

Mai9. 

Mich. 
Nev. 

New Hamp. 

N.J. 

N.C. 

N.Dak. 

Ore. 

S.C. 

lenn. 

Va. 

D.C. 



20 



Kan. 
Md. 
Utah 



Calif. 
Okia. 
Ttx. 
Wyo. 



S. Dak. 

Ohio 



Conn. 
D«l-« 
Iowa 
Me. 

Minn. 



Jan. 



Colo'. 
Ind. 
Mo. 
W.Va, 



N.Mex. 
N.Y. 
Penn. 
Wl»c. 



R.I. 
Vt. 



'June 



Alaa. 
Aria. 

Idaho 

111. 

Neb« 



Year/Month 



Subsequently 
funded - 
project!^ 



No Pvo)ecti 



Minnesota 



Sept. '71 



IlUnola 

N.Hamp. 



June '72 



California- 
California 
New Jeraey 
New To^k 



July '72 



Mafl(a£tKiail(« 
Michigan 



SOURCE: Abstract* of Cxemi^^^'y Project* In Vocational Educatioa; USHCW. June 1973. 



This policy paper served as a primary reference point for* both the funded projects, 
and USOE during the, first round. While the pi;oje<;ts varied in many respects, all 
were expected to comply with administrative and programmatic points made in 
this key document. • From a programmatic point of view, the paper specified that: 
"In order t o^ ch i e VQ ^ ax im um impact, the funds available for fiscal yeaf^ 1.970 
will be focused on programs or projects which combine, in one operational 
setting, all the following aSpect§: 



1. 



Pro^ 



:^nJor broad occupational orientation at the elementary and secondary 
school levels so as to incTrease student awareness of tlxe range of options open 
to them in the world of work. 



5. 



Provision for work experience, cooperative education^ and similar programs, 
rriakin^ possible a wide variety of offerings in many occupational areas, • \ 

Provision fot* students not previously enrolled in vocational pT*ogr>ams to 
receive specific training in job entry skills just prior to the time that they 
leave the school. (Some of these training programs might be very intensive 
and of short duration. ) 

Provision for intensive, occupational guidance and counseling during the l^st 
years of school and for initial placement of all students at the completion of 
their schooling. (Placeme;it might be in a job or in postsecondary occupational 
training. Placement should be accomplished in cooperation with appropriate 
employment services, manpower agencies, etc. ) 

Provision for the grantee or contractor to carry the program on with support 
from iiegular funding sources after the termination of the Federal assistance 
undeT ParUD of P,L, 90-576. (Feder^al assistance under Part D cannot 
exceed three years. ^ 

-8- 
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While this section of the policy paper concludes by anticipating "that other .program 
emphases may; be highUghted in future fiscal years," the focus for ali first round 
projects was to be as stated. Basecf^n three years of experience, a somewhat 
modified policy statement currently servos as a basic document for second round' 
projects. ' ' . ^ 

Concurrent \vitKthe selection by tjSOE of sites for the implementation of projC^ct^/^- 
f-unded from the 50% of Part D funds reserved for thb U.S. Commissioner of • 
Education, tha states awarded grants with their half of the Part D funds. In their-' ' 
use of P^rt D funds the states were bound ^ the federal legislation as to procedures, 
and purpose but not by the USOE policy pap^t^nd its designation of .ar.pa^/cif 
priority. As a result, the variation in the use of fund^ both wlthfn gnd jiet'<iveen^^^ ' 
states between 1969 and 1973 was considerabl'e.:v^ror'e:^.^^pi^ife, while some stales 
acbpted basically the same fupcling stHtt^gy.^as <lid USOE,* providing a few projects ' 
relatively lat-jre aiVu>urtts -^jjf moo.ey for a three year period, othei-^ funded a great 
many pifojoct^. tor a short duration. By the sam.e tWen, there vlas con.sideralJle 
..variation among- states with respect to the slpec^ic types of legiilatively authorized 
'activi^j^s which they supported. 

In summary, while there was considerableVariation among both state administered 
and federally administered Part D projects during the years between the passage of 
the legislation in late 19^8 and the end of the ^first three year funding cycle in June 
1973, there^were several key areas of commonality. The entire effort begSin from 
the Advisory Council on Vocational Education which in turn was based-on a 
comprehensive assessrnent jof the status of vocational education in the U, S», and the 
research and development efforts which were of note at the tirne. The Part D 
legisl^ion specified a general goal, two basic purposes, and a set of broad 
procedures which^were presumed to lead to accomplisTiing'.t'he purposels. The 
r^esponffibility for irr^plementir.g the procedures, i.e. ,* developing the speaifie 
techniques which would in combination accomplish the legislative purpose, was 
e^gplicitly delegated t^ USOE and the states presumably with the expectation that 
variations would occur. These variations, however, were to be Within the 
programmatic and procedural framework specified Ln*the Act. ' 

.4" ^ •■ ;' ■; 

B. Career Education: Its Rela tion to Part D ^ • 

~*j " ' — , , , • • 

The origins of the concept of career education may be traced batk several decades. 
As a phrase with special meaning to the educational corrvnunity,^ it is, gene.i^lly * 
accepted'^at its origin is the speech by then U.S. Commissioner oif fcducaiion, 
Sidney Marland, Jr. , Selivered in January 1971, at the Convention of National 
Association of Secondary School Principals, entitled "Career Education Now." to 
the months which followed, Marland made a series of speeches ^relating tb career 
education and expanding on various aspects of the concept. Vtve pojsitive respoB<^^ 
to his efforts was widespread. Career education became a maj^or thtust of the USOk 
during his tenure as Commissioner and evqntually merited legislation creating withilf 
USOE an Office of Career Education. The fulnctions of the Office of jcareef*^ Education 
include "providing for thej dern'onstration of the best of the current career education 
programs and fjf^acticesiHijy t,he development .and testing of exemplary programs andf 
practices. J., and 'developing State and Ipcal plans for implementing! career educafibrt 



pro'grarrLs. ..." * J"* . ^ . i\ 

As Marland used it and as carried forward in'tlie 1 ^^m^ldment^^ to the Elementary 
and Secondary .Edijcatibn Act which established arr 0?hc? Qf\aree^^*)E:ducation, th^^V 
term refers to a. s^et of concepts which have "been discusse*d1^y lea<ferS and. scholar^^ 
in vocational and;dther education for years. "-^^ In tracing th^ development of -v'^f* 
career education a^ an educational p4'ogram, leading advocate^both /<vithin the 
Office of Educationland without—^ cite the major contribution made/by the Advisfrry 
Council on Vocational Education of pulling together related res^^.'cii and setting \ ' 
forth specific recoiVimendation s . Given that, it was not until 19fr'that the term 4 



"career education" was popularized, there is no fnconslstency in the conclusion 
that while the Part D legislation and policy statements made no mentioii of it, the. 
projects funded under this section of the Vocational Education Act oon^liiiut^ son\e 
of the earliest efforts to operationalize the set of .coij^pepts whic^^re ndw 'associated 
wit.h career education, ..'^^ / . 

' J ,\ \: \-i ' ^ * ' / 

While 'tKe Part D progxa<m^ ^hich are the suljject of this stud£y rriay accurately be 
)nsidere4^ career education programs, it mu^t be noted tha£ they by no means 
^nstitute^all the federal and local effort over the*pAst several years. W»ith the 
dng of then Commissioner Marland, a great deal of activity was undertaken. 
ThelNational Institute of Educatfon provided relatively large amounts of funds to 
test Jou.r career education models in selected locations. Several states have 

<imade major commitments of time and funds to implement career education programs 
and several local schdol districts have Invested Jarge amounts of their own 
resourc^in implement^tidn. For its part, the U.S, Office of Education has made 
sighifieantSlnvestmenta m developing career education curricula and in support of 
career education research projects qther than thosX^ssoclated with Part D. 

' " ' ? • ' " ^ * i ' ' ' » ' 

" in« ^omrhary, while the Par^t D proj^^s and c^areer education have commoA origins 
and 'share many educational concepts^nd goaj^,.' i6 would not be accurate topresKirn^. * 
that the first round effort of Part D constftutes a fiar test 6f 'impleme^ntjng the j 
career ecJ'ucation 'concept, leather, the first round project^, par uicfular.ly tho^fe^ < 

"^supper ted from the Commissioner's share of the 'Part D Appropriation,* should be 
seen as an early approach to operatidnalizxng the conceji't in the local school*/ 

.sy^stehns, but only one of sever^such-efforts. v 

Evaluation bf Pa^rt D^Prdjects: Study Objectives ' " 

As the completion of the first rjound three-yejar funding cycle for Comnii^s ioner 'S 
share projects drew to a close^ the Office of Education determined that a thorough 
evaluation of the first round effort should be**undertaken. While it was recognized 
that an evaluation conducted at the end of the three year cycle could not directly 
address the long term objectives of tlie Part'D effdrts, it was presumed that much 
information which would be of assistance in the c6nf>*nued operation of the program 
could be obtained. 

More specifically, it was expected tha^^ evaluation of the Part D program would ] 
lead to improved implementation of the e^ort during the secpncl round of funding , 
and would contj-ibutef to the replication of s\K:cessful processes by local school ^ 
districts. Sinte thejpart D effort was so closely associated Vitlvthe concept of 
career education^ was also expected that inft^ination mi^lit be obtained in the 
course of the evaluation which would assist in pp^rationalizing the ciancept at the 
local level. 

Within the U.S. Office of Education the formal responsibility for evalu^ing Par? D 
i a that of the Office of Planning, Budgeting, an^ Evaluation.- In the spring of 1973, 
•OPBE prepared a statement of work for the Part D.eyalJ^tion which specified the 
fbijr following objectives: 



1. E^valuation of the effectiveness of Federal discretioDaayt^art D) projects and 
their, components where e'ffectiVeness ia defi^ieii as the exMSft tct which 

/ student outcomes attribu*table to project activities conform to the legislative 

;nte,nt for Part D funds. ^ " ' * , 



Performance of a cost atjalysia of each Federal discretiona^ry prpject covering 
the three-year period of Part D funding with focus on ;th^ relation ^f cost to 
student outcom>s§ and project process meeisures'; and per^brnikric^^of cost 
analysis for tliosk discretionary projects whose funding ha 91 terfnin^ted\ * 
comparing the period before termination of Part, D funding and jtjie |)eriod after 
J, its termination^^ ' - ' ' ,^ '\ 

^ - 10 - ' ' / " i ■ ' A 



Assessment of the influence of projects or their components on other S9t)pMs 
and school systems in their states, and isolation of feasibility of replicatt^r> ^ 
and determination of the applicability of project characteristics in other " * 
s,ottHngs . • ' ' ' \ * 

Performance of a descriptive study and' analysis of projects supported from 
state-administered funds, paying particular attention to their estimated 
impact; identification of different state allocation strategies to deterrKijie the 
differential impact of various strategiea. , ^ 



IS/ 



sJn June 1973, OPBE signed a contract to have performed the work necessary to 
5 a»j::complish these objeciives and work was begun in Jjuly. The work was to be 
a^^^omplished in two phases. 

Tht\irst phase of the effort was an "assessment of information available in anrfual 
and \iK^l reports of each discretionary project in tne first three-year cycle, on 
\file iV'VSOE, Washington, D,C., and discussions with USOE program managers 
Arnong ^f>^ purposes o-f this effort was to obtain the data base necessary to deveiop\ 

^^i^ig^And instrumentation for the field data collection and analysis phase of the . 
evaluait^n* .Implicit in the approach to the .evaluation was the assumption that basic 
, descriptive' data on most, if not all, projects was available through the project 
- annual reports .and the externally conducted evaluations" required annually of each 

fliar3,e<!l'3Sn the extenXvve review of materials available regarding state and federal 
P\^^^Q5I?^i}^ an ac^aly^is of third party evaluations, it was apparent at the* end of 
theyfatiSt several mW^-hs that uniform data on the projects did not exist. Similarly, 
it wVVlipparent that where evaluations addressed the areS^yof project irnpact on. 
participating students^ tK^ data produced was not amenable to cross-site comparison. 
There was^Utfle uniformity, in the indicators selected by the projects with regard to 
achievement' of the progrSm ^>bjecti-ves specified in the Part D legislation and 
related US,O.Ej::pfvU^y7^ --^-'"'T^ ^ 



Given the divji^r^ity^;^ the projects with regard to specific student-related outcomes 
as well as the eJqi^^teU divers ity in program processes, the decision was made by 
USOE and the contr^ctor\to develop a set of student outcome and program process 
hypotheses which w^J>^ie^logically to the legislation and Part D policy statements. 
These hypotheses ar$^ ^rese^ted in the context of the next chapter. 




BAScd upon an initial ;^alysis of dat.>| durih^l^^ ^si^n phase 

in this' reguid was not feasible owin^^'to the^ci^s^afti^ of time, cesources.^i^d available data The contract 



It was determined that an extensive effort 



^ amct\^ed accordin;«ly 
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CHAPIER III: EVALUATION FRAMEWORK 

• • ■ r- • 

The purpose of this chapter is to provide the conceptual framework for the study of 
the federally administered portion of the -Part D program (Sec. l,42(c)). The rela- 
tl )nship between federal activities and .ibjectives to local activities ^nd objectives 

\s discussed. ' Four bans ic program elements*imd three relationships ur pricesseb 
are identified and the^^etudy questions associated with them are presented. Dis- 

,cussion of the approach to the study of state administered Part D programs (Selc. 
142 (d)) is presented in Chapter XI, together^ with related findings.' V 



A. Overview 

The complete process of evaluating a program such as Part D requires the perform- 
ance of several activities including tiie staternent of program goals, the measurement 
of the extent to which the goals were achieved, and finally, comparing achievements 
to goals and making a judgment on goal achievement. In a very real .sense, it is » 
* this last step, making a judgment on goal ac-hyjvement, which is evaluation. The 

object uf this study was to measure prog ram^atjhievements and to analyze the results / 
as a basis for evaluation. In the sense used here, evaluation of the fitst round 
Part D programs is to be performed by the USpE an4>the Congress. 

This national study of the federally administe^red portio-^ of the Part D program wa4 
to focus on the' results of 50 different projects, located in 49 states and the District 
'•^ of Columbia, Each project had its local goals and objectives, and each had been , ^ 

•^h ^' independently evaluated in these terms. It was ^ condition of the Part D grants that 
each project arrange for such an evaluation annually and that the evaluation reports 
be submitted to USOE for review. 

For several reasons, it was explicit from the start that this study would not be a ^ 
, (^ompilation of 50 individual project evaluations. First, it was recognized that the r 
diversify among the projects in terms of locally determined procedures and objec-. ^ ♦ 
tives would make meaningful cross -projeqt summary statements difficult under the 
Taest of circun^stances. Second, it was recognized that the re was no consistent 
"^iramework used by the local project evaluators and that the quality of the evaliaations 
W^ried considerably regardless of the framework used. Finally, and most impor- i 
* tKnt, the study was to focus its limited resources on the federal effort and the 
deg^ree to which the entire Part D effo>t/realizeT7e^e ral legislative and policy > 
objectives. In this context, each projecit :55ias viewed as a component of the federal * 
effort and each was expected to cor.i^ribute to the realization of the feclerar goals. '| 
. - ' ' / 

Thus, while each project had its own iShort? range objectives and longer range goals, ^ 
the focus of this study was to be on the accomplishment of federal objectives of 
Part D and, infe rentially, on prQgres5 toward thr federal g*oaL Given this perspec- 
tive, it was to be an assessment of the extent to which the local projects contrifeutedy 
to achievi^ the purposes of I^art D as defined in the legislation and federal policy 
statements. As such, the study could be considered an ass^essment of any individual 
Part D proje'ct only to the extent that its objectives \ve re congruent with federal 
V obj e ctive s. ^ ' \ i 

The discussion presented in the preceeding chapter described and summarized the 
salient points of 
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salient points of the Part D legislation and its origins. In essence, the legislation 
provided for the expdnditure of.ifedjeral funds in local school ^stems in order to 
produce a specified se^t of activities which in turn would produce' an effect on stu- 
dents which was expected to contribute toward achieving the go^l of the .legislation. 



- 13 . 



The tyiJical relationship among ihe elements of the P^rt D program may depicted 
as in Figure 1 belrow. Many evaluations encounter difficulties by failing to differen-J 
tiate conceptually between fed^.ral and local inputs, and federally expected and localf 
activitie's, outcomes^ and gdal^ This important distinction is reflected in Figure 1 



and illustrated more fully by Frgure 2. 
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The, relationship presented in Figure 1 between federal and locai inputs, activities,, 
outcomes, and goals is mecuif to be illustrative of one of several logical possibili- 
ties. The actual *relationship/may be exlpected to vary across projects. 5^gure. 2 
below illustrates the five logically possible relationships between the federally . 
expected program outcomes and those of a, single Part D project. As suggested by 
the figure, the locally planned Outcomes ot a project operated with Part D funds . 
could: a) be the same as the federally expected outcomes, ^b) be only a portion of 
the federally sponsored outcome^, c) include all federally expected outcorries ^ithin 
a larger set of local outcomes, d) include only a portiorf of the federally expected 
outcomes within a set of broader outcomes, or e) be independent of the federally 
expected butcojpries. ' 




Federal outcome = 
objectives V 

Local outcome 
objectives ' 




41, 




FIGURE 2 

^'possible' RELATIONSHIPS BETWEEN PART D PROGRAM OUTCOME OBJECTIVES 
AND LOCAX PROGRAM OUTCOME OBJECTIVES 



\ 



The sa^pe set of possible relationships exists between locally planned activities and 
federally expected ^activities and between the federal goal and the local goal. While 
by definition a Part D project will utilize some Part D funds, in some instances a 
project may utilize only the funds *!kvailable thx)ugh Part D, and in others it may 
supplement these , funds. Although one might hypothesize a relationship between 
the origin of project resources j!>r inputs and the origin (i. e, , federal verus local) 
of project activities and outcpmfes, pure logic does not permit presumptions in 
.this regard. * 
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B. Evaluation Framework Applied to Part D 

Given the view reflected in Figure 1, the Part D program may be considered for the 
purpose of this evluation as haviXg four basic elements ) Federal Goals; {Z) Fed- 
eral Inputs; (3) Federally Expected Project Activities; and (4) Federally Expected 
Student Oytcomes. ^The elemcnt;,s build on each other and are linked by three dif- 
ferent ^but cumulative relationships. Cach of thes,eua^_jles-Cjrabed br iefly^below. 
The description of the four program elements is followed by a dis/:ussLor. of trie 
three sets of relationships^ . tf' ' * 



1. Federal Goal 



As stated in the federal legislation, the goal of Part D of the 1 968 jamendments 
to the Vocational Education Act was fo "reduce the continuing seriously high 
level of youth unemployment. " 'xhe legislation doesinot explicfitly state that 
Part D alone will result in a reduction of youth unerAployment. ' Ra4iier, it is 
^ implicit that Part D represents one of several steps Vp be taken in reaching 
this goal. ^ ^ • ' * 

the legislation indicate s|?^that an effdrt designed to make schools as effective 
in meeting the job preparation needs of students not going to colleg*=* as Lixey 
are in meeting the needs of the college-bound student would* contribut^ to the 
realization of the goal of reducing the high level of youth unemployment. The 
legislation states that this effort ;pequires developing '*new ways to create a 
bridge^tween school and earning a living for young people/" in part by promot- 
itig "... cooperation between pu^jlic education and manpower agencies " 

Fede ral Inputs ■ , 

From^ the legislation, the Federal Register, .and the USC^ Part^D policy! paper, 
it can be inferred that in order to make school more relevant to jthe job prepara- 
tion needs of students, projects were to be funded which aimec^ at broadening 
oth occupational Aspirations and opportunities^ for yquth, especially those with 
ademip, socio-economic^ or other handicaps- \ ' 

ToUhis end the act\auJhori2ed the appropriation of up to $222, 500,000 for a\ 
perW of four years. 6f this amount, $67 million was actually appropriated^ 
over four years Witl? approximately ^3'3, 500,000 devoted to the federally ^. 
admini^-tered projects under Part D (Sec. 142(c)). The funds actually appro- ' 
priated Represent the resCruices allocated toward reaching thd goal ux reduced ^ 
youth unemployment through Part D. lA other words, the federal funds alloca- 
ted, ii^ accordance with a legislatively prescribed formula, constitute the 
federal resources or inputs into each Part D project and in total to the Part D 
program. * 

Responsibility ^or administering these inputs was delegated by the Congress to 
the U.S. Office of Education, and from there to the USOE Division of Vocation- 
al and Technical Education (DVTE). Operationally, it was the responsibility 
of DVTE to insure that the legislatively available resources were utilized.as 
planned. 
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,3. - Federally Expecl:ed Project Activities 



Ideally, inputs are made with the expectation that outputs will be produced: In 
this case, grant awards were made with the expectatfbn that certain activities 
would occur which would. lead to achieving student outcomes and ultimately the 
achievement of the program goal. These activities constitute the planned fed- 
eral-activities at the level of the local school district (LEA). While these 
activities typically will be contained within %e field of local activities, , the 
relationship of the federal activities to local activities may vary from project 
to project. Iii^.some locales, the LEA may focus ^entirely on the federally 
expected activkies, while in other place ^ only selected federally expected 
activkes may be^ addressed.^ 

In the Qontext of Section 142(c) of Part D, the "Commissioner's share" effort, 
the project level activities which federal legislation and USOE policy* specified 
should result from federal expenditures were that there be: v 

• in each state a formal^y^designated project functioning in accord with Part D 
0 policy and funded at an annual rate of between $100, 000 and $200,000; - ^ 



• in each project, ^ome level of activity directed toward occupational orienta- 
tion at the elementary and secondary school levels; 

• in each project some level of work experience or cooperative education 
activities; ^ ^ ^ * ^ 

• in each project an effort to provide intensive job entry skill training to stu- 
dents just prior to their leaving school; . v * 

• in each project an intensive guidance and coinseling program for students^ 
and ^ ? 

• in each projd.ct the initial placement of* students leaving school. ^ 

In additiorKto these activities, there were activities meiitioned in the l^islation 
but not re'qujiredjin the USOE policy paper, '^hese activities are: 

• activities for young workers released frobn their^obs on a part-time ba^is 
for the purpose of increasing their educational attainment; , 



activities encouraging the exchange pf personnel between schools and other 
agencies, institutions, or organizations' participating in activities to achiev 
the purposes of Part D, including manpower agencies and industry; and . 



• activities designed at the secondary level to motivate and provide pre- 
professionaL pre pa ration for potential teachers of vocational education. 

4. Federally Expected Student Outcomes 

The object of expending federal fvinds to produce a set of program activities at 
the school district leVel was to produce a change in students which would coh«- 
tribute to attaining the federal goal of reducing the rate of youth unemployment 
Since neither the legislation nor the federal policy statements specified the 
student outcomes which were to result /rom implementing Part D projects, it 
was necessary in designing the evaluation to develop a set of expected outcomes 
logically related to the federally expected activities at the project level. ^ The 



* USOE Policy Paper governing first round Part D Sec. 142(cKprojects: AVL V70-1. 
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implicit assumption was that if these objectives were produced they\would con- 
tribute toward the realization of the long-range goal of reducing the high level 
of youth unemployment. These outcomes, when measured and judged attributable 
to projectt^activities, ^ould serve as indicators to the degree to which progress' 
had been made toward the realization of the long range goal. 

The expected student outcomes were spepfied jointly by the study t^am and the 
two salient units .of USOE (DVTE and OF^E). The set of- specified outcomes 
which emerged frgm the interaction was not intended to-he exhaustive. Rather, 
the final set constituted those outcomes which were considered reasonable 
expectations of Part D projects a,dmihistered in local school' districts relying 
primarily 6n federal funds. These objectives are as follows: • 



Student participants will be able to identify a gr.eatqr number of occup9,tions 
than non-particip^ants. ^ 



• Student participants will dennonstrate m5re familiarity with tasks and func- 
tions associated with selected occupations than non-participants. V 

• Student participants^^U be more familiar with the requisites associated 
with employment in selected occupations than non -participants. 

• Student participants Will score higher on pre-vqcational, job readiness .tests 
than'non -participants. 



• .Student participa rits will indicate more positive attitudes toward employment 
than non-particif ants. 

• The variety of careers being Considered by in&ividual participtiting students 
will be greater than the variety for non-participanfcs, <^ 

• Student participants will indicate more positive attitudes toward guidance 
and counseling than non -participants. 

• More student participants will indicate having a ^career plap than non- ^ 
participants.' <f - * 

• More student participants will include further training or education as a 
portion of their career plan than non -participants. 

• More student participants will cite their career preference as their expected 
career than non-participants. 

i 

• The number of student participants citing vocational education as their 
future career will be greater than non-particlpante. (Note: This outcome 
was based on language in the legislation that did not appear in the USOE 
policy paper. )L , 

9 Students will evidence behavioral gains (e. g. , less tardiness, fewe r^ absences, 
fewer disciplinary problems, etc,^) as a result of participating in career ed- 
ucation activities.'' * - :i 
f 

• A greater number of graduates within the past 12 months who have partici- 
. pated in the program will be employed full-time or in further training than 

non-participants. 



Rolatiunship Onv: -Federal Inputs Project Activities 

As indicated, ^the federal resources or inputs were expected,±a result in a set 
of specific activities carried out ih at least one school dismct in each state. 
Program activities in tjus context are understood to include methods and tech- 
niques employed in producing the prog mm outpvtts, i. e. , student outcomes. 
It lb. presumed that the relationship between inputs and outputs is considered to 
be a manageable one, which is to scvy that adjustments in the inputs will alter 
the outputs. These adjustments usually are, in fact, changes irv,pr6ject activi- 
ties. . . ' , • \- 

T^'picMly, an evaluative study first takes^ s ome measure of program ir^puts and 
outputs. Oilce this ;s accomplished, it is necessary to analyze and interpret 
the fiiidings in order for them to have meaning and uiility. Taken separately, 
there are four possible findings; (1) the inputs occurred asVl^LnneS; (2) ^hey 
did not; (3) the outputs occured gts planned; aifd ^(4) they did not. Once it is 
knoun whix?^ of these findings is the ca.se it is also necessary to understand 
the relationr^hip of the inputs and the outputs. As indicated in Figure 3, t^ere 
are four possible combinations. >^ ' ^ ^ 





Outmits occurred as planneti 


Outputs did not occur planned 


Inpirt^ occurroJ as pLmned 
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RELATIONSHIP OF INPUTS TO OUTPUTS 



Cell I indicates that both the inputs and outputs occurred as planned. In this 
situation the analysis focuses on program activities in an attempt-to discover: 

(a) what facto ifs have been important to success? and (b) are there l/etter oj: 
more efficient ways to produce similar results in the future? 

In Cell II, the planned inputs occurred but the expected outcomes did not result. 
When this happens,' it is a function of evaluation to det'^^rminc the reasons for 
foilure in order to avoid siixxllar failures in the future. Once again, activity 
x:onside rations are important. It may be determined that changes in method 
lor technique will correct the problem. It may also be determined that activity 
/ factors are not the problem. The level'of inputs simply may not be sufficient 
/ to produce the desired results. In the worst case, it may lie discovered that 
the inputs are themselves inappropriate to produce the desired outcomes. In 
the first instance, the solutinn is essentially a problefn of Improving the man- 
agement of the inputs. In theSlatte r •case policy -cons ide rations are involved in 
changing the planned inputs. 

Cells III and IV represent circumstances where it is determire d that the 
planned inputs did not occur. In both cases activity considerations are not 
as significant as in the cases already discussed. Cell III depicts an instance 
of the desired outputs resulting from less investment than 'planned. The 
oBvious questions to answer here are (a) "was the plan faulty and do the desired 
results occlir from a lower input level than was felt necessary?"; and 

(b) "would the outputs have oc|::urre4 in the absence of the inputs?" Both 
questions relate, of course, to policy considerations. At least theoretically, 
the production of planned outputs, without the planned inputs occurring, should 
be a rare occurrance. Usually, when the findings reveal that the planned in-* 
puts we're not made, the reasonable expectation is that the planned outputs 
also would not be produced, as in Cell IV, 
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The relationship between federal inputs (defined as the Part D Sect 142(c) 
fxiHG^ available to a project) and the specific* results they were expected to 
produce concerns the manner in<vhich,the federal government dispersed the 
r;art D filnds and the manner in jyhich the local projects complied with the 
basic provisions of their Part D grant. More specifically, the set .of primary 
factors governing the relationship betw.een federal inputs and project actjlvities 
maybe chaivicte rtzed as program management. Broadly, the factors investi-* 
gafed were: * 

• the stated objectives of the projects; 

• the nature of the organiz5kfions receiving Part D Sec. 142(c) funds; and 

• compliance with federal policy with respfect to pr9ject management. 
Relationship Two; Project Activities to Student Outcomes 

The basic logic with respect to interpreting' the^data relqited to this set of pro- 
cess factors *is the same , as described above. Essentially^ if activities and 
student outcomes occurred as planned, the focus would be on isolating the 
factors associated with project. succe^- and on detdrmining if the most cost 
effective approaches were used. ^pt^tfte other hand, if activities were not as 
pla,nned, positive levelp of studtjnt outcomes woxild not be Expected and focus 
shovild be on explaining the unexpected lev?l of activity rather than on unexpec- 
tedly low student results. ^ 

Based on the a/sumption at the start of the study that project activities general- 
ly would conform to planned federal ^prog ram activities, the following were 
determined jointly by DA and USOE as the project tneatmenj^factors or indica- 
JxDrs most likely to explain thfe student outcomes of the. Part D program: 



Participating students will engage in more career familiarisation curriculum 
activities than will non-participating students. 



• Participating students will engage in more out-of-class career 
activities than will non-participating students. ' : 

« ' / 

• Participating students will be exposed to more career resource people in 
the classroom than will non-participating students. ( 

c A greater nui^iber of participating students will be assiste^ in securing jobs 
or in entering work experience programs during the scho(^ year than will 
non-participating students. 

• Particip^^t^rlg students will receive more' occupational guidance* and cotmsel- 
ing during a school year than will nori-participating students. 

• Participating teache rs will encourage students to consider careers in voca- 
tiorial education. 



Relationship Three: Student Outcomes to Federal Goal 

In essence, the relationship between the goal of reduci^ng youth unemployment 
and achieving the student outcomes associated with Pa*rt D is hypothetical. 
Also, an analysis of this reljitionship cannot be expected to provide a suffici- 
ent explanation of the extent to which the goal ^was attained. Clearly, the 
nature of the job fnarket, f6r example, has as much to do with the level of 
youth unemployment as the skills and, attitudes of the youth. 

^ / 

s 
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Given the nature of tiis relationship and the judgment by USOE that three years, 

of program operation^ was too short a period to expect- full impact from a 
program operating at the elementary through high*school grade levels, this 
study does not address eithe^the attainment oif the long range goal or the rela- 
tionship between the goal and student outcomes. -\ ' 

C. Summary and CotTclb^^ns 

In tl^ first section of this chapter a general frameworkwas presented for des- 
cribing and evaluating a federal pTOgrkm, such as Part D, which is implemented 
through grants to agencies at the locallevel. The framework, Figure 2, ^ 
identifies four program elements and three distinct relationships. 

The program elements and processes were discussed briefly in SectionJB above. 
It was noted that for this studv^f tfie first three years of the Part D progrcim 
qu^^stions pertaining,^) the piofram's* impact on the long range goal of reducing *^ 
youth unemployment could not be addressed. Figure 4, ^elow. illustrates the 
framework as it applies %t6 this stu^y^ In*this illustration the jkhree program 
elements and two sets of process factors around Vvhich Part D data were collec- 
ted and analyzed arp identified. The figure ajso* illustrates the relationship 
between ^a number of chajSters In this 'report. Specifically, Chapter V addresses 
federal inputs or resources. Chapter VI addresses project level activities or. 
the extent to which the federal resources resxxlted in the expected activities at tl^ 
project level; Chapter VII addresses the relationship between the management of 
federal resources and achievement of project level activities. In Chapter VIII 
the student outcomes of the Part D program' are presented, and in Chapter IX the 
relationship between these outcomes and project level activities is discussed. 
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In Chapter IV, which follows, thenrocedures use<| to collect and ajialyze the 
Part D Sec. i44(c) study data are discu^sejl. Methods and findings pertaining 
to the descripMve study of state administered projects (Part D, Sec. ^42(d)) 
are presented in Chapter ^ 
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CHAPTER IV: STUDY METHODS 
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, * A . Obtaining Backgroutid Information • ? - > • * 

The general plan for this evaluation began with a process in which background* 
information concerning Part D^projecEs was to be obtained through review of 
^various documents in the files of DVTE. It was expected that grant proposals, 
'third party evaluations, interim reports, USOE site visit evaluations, corres- 
pondence, -and other grant award materials would provide information^describing 
the character i»;s tics of Exemplary Projects and of the students participating in 
them. Information regarding the following topics was sought; 

1 . Characteristics of Ej^mplary Projects 

• Objectives and major components of projects; 
^ * 

• Descriptions of the scope of each major component or activity; 

.1 \ 

• Identification of materials and techniques used; 

• Organizational and management structure of project; 

• Size, background, qiaalif ications , and roles of project staff; 



Linkage of project to local and state educational decision-making 
structures; * ' v 

Linkage^ of project fo manpower agencies, employment service 
agencies, and employers; . • 

Summary of project costs by^'category and by source of support; 

<3eographic and economic setting in which project is'located;* 

Summary and assessn\ent of third party evaluation; and • 

Summary of major changes occurring in project since initiation of 
Part D funding. 



2. » Student Charaicter istics , ^ 

^ • Number of students participating in project out olf the total eligible 
populatiQn of school or school system, if project is systfemwide; 

c 

t 

• Breakdown^of partic ipating students by curriculum (general, voca- 
^ tional, academic), grade level, an4 component or activity (required 

in policy pacper or in addition to thd^e required); and 

• Number and characteristics of participating students placed and 
summary of followup information. 

This information was to be utilized (a) in developing a project-by-project 
^ description^^f^e federally administered projects which could lead to the 
developnQient of a typology of these projects; and (b) as the data base neces- 
sary for developing the research design and instrumentation to be used in 
the s tudy . \ 
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The review indicated that inuah of the anticipated information was either not 
available, or was not available in a fornn that pernnitted fulfilling the tWo 
objectives indicated abave . The alternative procedure adopted was to seek 
consistent and detailed information as to the size and scope of the individual' 
projects by me'arft of a mail survey, and to Qollect more descriptive data 
during the study's onsite visits. 

In August 1973, eacii of t^e 50 projects to be visited was sent a set of forms 
which were to be completed and 'returned to DVTE. Each project was re- 
quired to report the number of participating and non-participating students ^ 
and teachers by grade and school for participating schools in the school 
district and the number of studeits and teachers by grade for non-participat- 
ing schools. In, addition, the projects were asked to report the number of 
12th grade participating and non-participating students, enrolled in academic, 
vocational, and general curriculum programs. 

« 

By the end of September, approximately half of the projects had responded 
to the DVTE survey and by December, 34 projects had responded. During 
December a portion of the requested data was secured from the remaining 
projects by telephone. This background information Regarding participating 
teachers and students reported by the projects was next used in planning the 
logistics of the field operations. (As will be elaborated later in the report, 
the quality of the information varied considerably across sites with regard 
to the. number of participants but was generally accurate with respect to the 
, nurjfiber of participating schools. ) ° ' . 

"^/^ ^ ^ 

B. Study Design 

During the first three-year cycle of Part D funding, 6l projects were supported 
in the 50 states and the District of Columbia. Typically; one Part D, sec. 142tc) 
project was funded in each state. In several cases, the first funded project -s, 
terminated at the end of one year and another project was started in its place. 
In four states, more than one first-rou^d project was in operation at the same 
time-. The projex:ts included in this s^udy were selected to nYeet tlie following 
criteria: First, one project in each state with a first-roufid project ope^rating in 
FY" '73 was to be visjted; as Hawaii had no such project, 49 states and'bie District 
of Columbia were visited.* Second, each project visited must have proposed to 
focus activity at the elementary s^choo'l, junio/ high*school, and senior high school 
levels. Third, If more than one project met the second criterion, the first 
project: funded was selected. ^ - / 

While in the majori^ of cases'the projecTb^compassed only one or a portion^of-^ 
one district, in several states the Part D, Sec;- 142(c) project encorrgi^ssed ^ , 
several school districts. In all but two cases these "aistxicts were treated in the 
same fashion as a single district. Where dis^tricts were contiguous, the set of 
participating districts was defined as^one ''p"roj.et^y^'foa:.the, purpose. of drawing the 
student sample and obtaining other LEA^^ecifii ih^iin^i^^'^jnjhe two cases 
where the project included^d is contiguous districts, tin^E^^esoltr'ce constraints 
made it impossible to select students Randomly from tbe entire project. In the^e 
two cases, the particioating district considered thermos t^tstanding by the Parl-^ 
D, 142(c) grantee (the State EducatiQn•^'g£Sl£irJ[^^^^ 

Within each of the 50 projects sele<;;ted, the broad design called for administering * 
questionnatres and tests to both^articipating and non-participating students in 
grades 6, 9, ^and 12. These gVadfes Were chosen because they represented the 
terminal yeairs for the differing Curricula and objectives generally assigned at the 
elementary, junior high, and semQT high levels; they would thus be expected to.be" 
the grades wliere students had received the maximum impact in each of these 
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three curriculum segments. yQuestionnaire-^ wbr^e also administe'red t^s ' 
participating ai|(] non-participating teaV^^/s ^ ,^nd to 'participating schoo'l catin- 
selors. In'addition, int'e rvie\Vs\Y^e re coivducted wifch local proj'ecf staff and 
SEA staff. . ' , ' >' . , 

^ ' V , 

Preliminary' inquiries Indicated that most pi'ojects were not able to identify 
precisely eithe;r the students who were "pa^iclpants" in the program, or what"'.^ 
constituted "parti,c jpation. " , Cons iderable effort was devoted Jto finding satis- .\ , 
factory operational definitions c5f these terms, since participation was the key* ^, y 
factor governifig the-gtu'dy design* Asr a result of this effort, the following 
definitions were adopted: . ' * - 

^ • A participating teacher was defined as a teacher who: 

Received project-supported ^raining., or 

Utilized project- supplied materials, or - 

^- . Utilized project staff in class orjin developing course activi^ties 
(a' Pjroject staff member was jdeflned as a person whose salary 
wa's paid in whole Or in part by Part D(c) funds ), or 

Began engaging in course activities advocated by the project staff 
since -the inception ©f the project,^ and was considered by project 
staff as a "participant.'^ .• * 

' ' . * 

A participating counselor wa^s defined as a counselor who: 
>- t / /' ' 

Received project- supported training, or 
utilized project- supplied/materials, or 

Received assistance or- infoi*mation from project staff in the ilnple- » 
mentation of guidance arid c^ounseling activities, 'or 

Alte red s ome aspect,s of \iV& guidance arid counfeelir^g program such 
that it conformed ta activities or approaches advocated by the 
project staff since incep^tion of the project, and was consi^e red'%by 
project s.taff as a^.^'pa^rticipant. " - , 

*^ * ' ; • . . y 

Participating students were'define d as follows: < 

6th grade participating* students were current 6th grs^de students! of 
a participating teache^r. ( , 

^\ «' ' 

9th grade participating sti^dents were current 9th grade students of 
at least one'pa rtic ipatin^ teacher; or students who had received 
guidance/counseling service from at least one participating coun- 
selor or staff member: during the current school year. 

- ^12th grade participating students were current 12th grade students 

- wb6: 

7' -- were enrolled in a project-supported Nytfrk experience program; 

-- were enrolled irf a project'-^st^pported "job-entry skill trainin;g" 
program, and. ve^e not iir a'.work experience program; or ^ 
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were talight' J>y at least one participating teacher or^-staff 
membe^, apd were not in either a wprk experience p.rogram or 
arjob entry skill training program; or - 



were. not in one of the abovs groups and had received guidance/ 
* counseling service (including job placement) from at least one 
participating counselor or^ staff member during the current 
^li^ol year^o^r ' ' ^ . 

had been expo^d^ during the current^h^ol year, to a^project- 
suppose 5i activity otl>er'"Sian thos,e-^ted above, and were. -no't , 
in Qj)^ of the abpvis' groups . • ' • ' • 

.-^^ .'-"^ ^ 

Th^ two^e-finitions for the 9th grade are not mutually* exclusive, and no distinc- 
tiorLW^nxadfe betw'een them, lot sampling purposes. The 12th grafde'groups, 
^owevei:,^ .were defined Ttd yield mutually exclusive groups, and s^a^npUng and 
* coding 'procedure's- We re designed to preserve these groupings for analytic pur- 
. -po^es^^^^s; i^icated in Figure 5, below, the groupings are defined in a hier-' 
^J^pcW^aT manner; i.e. , the students in the Xirst groUp may also' have received the 
s'ervices cited for the other groups, the students in group (b) may also liave 
received the'§ervices cited for 'groups (c), (d), and (e), etc* ' ' 

. ' . ' r' 

TJie approach to stratifying the 12th grade participants was developed in response 
^ difficulties esncountered during pre-test visits to Part D projects. During these 
visits jt was found that high school students often partici|iated in several- project^ ^ 
activities, but typicallyjjrojects had no records of the number of activities in 
which a student participated, and that names Qf individual students were avail^i^le 
by d'lajs but unduplicated lists of participants had to be specially constructed for 
p this study. ,^he 12th grade strata were defined so f^^P make feasible the con- 
struction ol unduplicated lists of participants from.which samples were drawn. 
Edch student sampled was asked questions that wer^' developed separately-. for 
each project to determine whether he had participate^ in the activities in addition 
to those primarily associated with his stratum^. . ^ ^ , 
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The original design. Jiad called for a four-way class ification .of 12th graders, on- 
the basis of "course of study, " i.e., academic, vocational, gene]5"al, andout-of- 
scijpol students. This approach proved to be infeasible, however, since most 
projects were not serving out-of-school students and most LEAs did not classify 
students into the other three groups. 

The basic design called for the administration of tests and questionnaires to both 
participating and non^partic ipating students at the 6th, 9th, and 12th grade levels. 
In addition, questionnaires were administered to participating and non-participat- 
ing 6th grade teachers as well as t;o ^participating teachers in grades 1-5, 7-9, 
and 10-12, and to counselors. (T}}^:hon-^pa rticipant teacher questionnaire was 
similar to, but not identical with,J£h€ questionnaire completed by therpartic ipat- 
in^ teachers. ) For ease of exposition, these groups of non-participants will be 
defined m the context of the discu&sion of sampling, which follows. 

C. Sampl'ung. Procedures : Students 

The basic principle foUpwed in the s*ampling procedures was to insure that within 
each project, each pf rtic ipating sti^Jjent within the 6th grade, 9th grade, and 
each of the five s^trata of the 12th grade ha^ the same chance of being included in 
the sample as every other student in the same stratum. In addition, the number 
of students to be sampled in each straturn was to be 5% of the participants in that 
'stratum and no fewer than 30 students. Stiidents were to be selected randomly, 
without regard to classroom units; thus the student became the unit of analysis. 

Due to logistical, time, and cost cons idera^ons , it was'decided to draw ^he stu- 
dent sample for each stratum from no moi>g^?lhan five schools. In projects having 
five or fewer schools involved in a given stratum, a proportional sample was 
drawn from each school, i.e., the number of pa rtic ipating students sampled from 
each school was proportionate to that school's representMion in the total'number 
of participating students in the stratum. In projects* having more than fi^ve schools 
involved in a given stratum, five schools were selected by using a weighted ran- 
dom procedure , i.e., each school's chance of being selected was proportionate 
to that school's representation in the total number of participating students in the 
stratum. An equal number of students was then drawn from each of the fiv^ 
schools selected. Both o^ these methods -- the proportional sample involving 
all schools when there was no school selection, and the weighted procedure for 
selecting schools when there were more than five schools -- gave eacli participat- 
ing student in the project stratum an equal chance of being sampled. 

In both methods, total sample size for a stratum was calculated on the basis of 
,5% of the number of participants ^in that stratum, iwith a minimum of 30. Where 
a stratum had less than 30 pa rtic ipants in the project, all of the. students were 
used in the sample. ; 

In the four cases where the number of participating schools increased substanr 
tially during the third year of the project, student sampling was limited to the 
schools which participated f-or the entire project period. For example, in one state 
where 11 high schools were participating during the third prpj|^:t year, the student 
sampjle was drawn from one high school wii;h which the project had been associated 
sii}ce the first year of operation. ■ ' 

'%^e field procedures developed for selectingfth^ specific students involved the 
use of a table of random numbers applied tojro'sters of students in each student 
group. Detailed instructions were provided to the field workers to assure success- 
ful implementation of the sampling plan. Table 2 presents a summary of the 
sampling approach used in the 50 projects that were visited. 
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T 

G 


r a d e 


(stratum) 






S a. m TDlinCf Annrodch 


6th 


9th 

7 til 




l^th 








(1) 


(2) 


(3) 


f A \ 

(4) 


(5) 


(6) 


(7) 


Proportional approach ^pprojS, 5%; 
6p0 or more participants) 


7 


4 




1 


1 


-- 




Fixed number approach (apprbx. 
30; less than 600 participants) 


39 


39 


21^^ 


15^/ 






3^^ 


None sampled; No participants 


2 


4' 


28 


32 


23 ' 


• 

43 


43 


None Sampled: Administrative 
reasons-^ 


2 




1 


2 


1 


2 


4 


Total 


50 
t 


50 


50 


50 


50 


50 


^50 



TABLE 2 



" Due to the small number of students in the stratum and scheduling difficulties, m eight of these projects fewer than 
25 students were tested. 
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Due 'to the small number of students in the stratum and schedulmg difficulties* m five of these projects fewer than 
25 student^ were tested ^ 

^ Due to the snail xiurrKer of students m the stratum and scheduling difficulties, in seven of these projects fewer than 
25 students were tested 



4/ 



5/ 



^e to the small number of students in the stratum and scheduling difficulties, in three of- these projects fewer tha 
25 students were tested 



Due to irregular school attendance, the number tested in twp of these projects was less than 10; because of the special 
nature of all three projects, their results ar« not included in general analy^s. 

^ The absence of participants in a stratum does not necessarily imply the absence of a project actmtyj at the 12th grad^, 
for example, in 12 projects all counseling and guidance was provided m conjunction with other activities. Ajso, whert 
projects had completely terminated an activity, students could not be tested. 



6/ 



7/ 



In one project no testing at the 6th, 9th, and 12th grades could be scneduled; in two projects, no 9th grade testing 
could be scheduled but 6th and 12th was possible: and in another, no 12th grade testing was possible although 6th and 
9th was In the case of one project, so few students at the 6th grade level actually were tested that they were - 
excluded from the analysis In the case of stratum 7 (12th grade 'bthet"), either students did not attend school on a 
regular basis, or they could not be identified. 



For each group of participating students selected for testing atxeach project, a com- 
parable group of non-participating students was selected, if avsnlable. These 
"^control" students were also selected randomly (not on a classroom basis), and 
Were taken from the fSllowing sources, in or.de r of preference: 



V 



same schools as the participating students; this source was used only if 
the school did not apply any special selection criteria (e, g. , academic 
achievement) .for placing studen^ into classes taught by participating 
teache rs ; V ♦ 



i 

i6-- 
A7 



Th 
nil 



fiiitiTtMU Schools wirhin the samt- si^hool dib^ ru L;, selo c tion' of schools was 
based upon the judgements of school officials and study staff concerning 
income levels of parents served by the schools, the ethnic composition of 
the student bodies, and the size of the student populations. Where these 
factors were judged to be equal, preference was given to schools located 
relatively close to the participating schools. The same control school was 
util/zed for two or more of the participants' schools if the latter iwe re 
si/hilar to each other with respect to the above factors; '><S^tlcv>w. 

different s^ool district; comparable schools were selected outside of the 
pa rticipdlirg school district, using the same criteria indicated above. 



sources of the control students at each grade level are shown m Table 3. Over- 
control groups were available for 138 participant grdups; .twenty-four we re from 
the same schools as participants, sixty-six from the same school district, and 
foriy-eioht ucre from different districts. ^ 



TABLE 3 
SOURCE OF CONTROL STUDENTS 



Source?' of Control Students . 


Number of Projects 




Total 


6th Grade 


9th Grade 


12th- Grade 


Work Expe r. 


Other 


Same schools 


24 


7 


5 


7 


5 


Diffe rent Schools, same LEA 


66 


24 


24 


4 


14 


Different LEA 


48 


14 


13 


9 


12 


Not Available 


f 

10 


li/ 






6^^ 


Not Applicable , 


sz 


4 






j2 


Total 


200 


50 


' 50 V 


{ 

50 


- 50 



~ \lichigan 
"^Michigan 
Vermont 
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lowa. Mass- . Michigan. Nebraska. Utah, and Virginia 



As shown in the table, two types of control groups we re^utilized at the 12th grade 
level. If the project had a work experience/cooperative education component, the 
control group for these participants consisted of comparable students eni^olled in 
a work e:<:perience program that was'not supported by the project; such control 
groups were located for 20 of the 21 fJrojects that had a work experiencte component. 
If there were participating students irt any of the other 12^th grade strat^/ the control 
group consisted of comparable studentjs not enrolled in a work experience program; 
this type of control group was utilized in 32 projects. Both types of cot^trpl groups 
were used in I6 projects. 
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Table 4 presents a summary of the number of participating and n©n-participating 
students tested. 5ome participating students we re tested in each of^he 50 projects 
visited. However, because of the small numbers in one project and the special 
characteristics of.tliose tested in another (both were from the 12th grade "other" 
stratum), data from only 48 of the 50 projects were used for most Student outcome 
analyses. ^ 



The comparability of the participants and the non-participants was checked with, 
respect to two characteristics: sex and ethnicity. The percentage distributions 
of th^se characteristics for each group of students are shown in Table 5., 



. TABLE 4 

TOTAL STUDENTS TESTED BY STRATUM: PARTICEPANT/NON-PARTICEPANT 



'Category 



Grade (stratum) 



6 th 
ill. 



9 th 



12th 



(5) 



Total 



No. Participants 

No, Projects 

Mean Participants per 
Project 



1,447 

.46 



■T^31. 5 



1,391 
43 

f 32. 3 



511 
21 

24. 3 



427 
16 

26. 7 



712 
26 

27.4 



105 
5 

21. 0 



4632 
50 

92. 6 



No. Non-Parti^pants 



No. Projects 

Mean Non-Participants per 
Project 



1,318' 



45 



29. 3 



/ 



/ 1,297 



42 



30. 9 



446 



20 



22.3 



499 



15 



33. 3 



483 



23 



21. 0 



(214^^ 
5 

(42. 8) 



4043 



46 



87. 9 



1/ 



The numbers t^ted is these projects were: 5, 7, and 27. Bccatue of special eotrafll^fequlrements for these 
coxnponej^ts. no comparison groups could be found and the results from testing these students are excluded 
most anal>'ses. - ' : 
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The total number of non -participants used for comparison v^rith Group 5 participants was 7S9; 276 were also used 
for comparison with Group 4 participants. ^ ^ 



" The total cumber of non-participants used for comparison with Group 6 participants was 214^ 53 of these were 
also used for the Group 4 comparison and the remainder were also userffor the Group 5 comparison^ 
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Statistical comparisons between participants and non-participants with regard to 
the sex distribution, by means of the chi-square test, indicated no significant dif-~ 
ferences except in the job entry skill training group, where 66% of the participants, 
but only 50%^of the non-participants, w^re males. For the ethnicity factor, 
however, the tS^sts indicated. s ignificant differences in four of the six student groups. 
As a result of this finding, and because it was believed that ethnicity may in fact 
have beer> an unwanted inflAi^rtx^ing fac-tor in the results obtained for other variables, 
it was necessary to institute a weighting procedure, prior to making comparisons ; 
between participants and non-participants, which would eliminate the differences 
in ethnicity between the two groups. Using this procedure, in which the response 
of each student for any variable is weighted in inverse propoftion to his ethnic 
group's over-or under-representation in the total, any statistical comparison be- 
tween participants and non-participants became, in effect, a comparts dh between 
two groups having identical ethnic distributions. This approach w§s followed in all 
comparisons between total groups of participants and non-participants in each o^ the 
SIX student groups. No weighting procedure was used in analyses within projects; 
m the few instances where large ethnicity differences occurred within a project 
they will be cited in the discussion of the findings. 

1 

D, Sampling Procedures: Teachers and Counselors 

I 

In accordance with the study design, in each project information was to-be obtained 
from selected pajrticipating teachers, non-participating teachers, and participating 
guidance counselors. The sample of participating teachers in each project consis- 
ted of: y . o . . 



each teacTier of the selected 6\h grade participants; 



approximaltely 10 teachers in grades 1-6 from the schools from which participant 
students were sampled (in addition to the 6th grade teachers above); where 
grades 1-6 were not in the s^gpe school(s), teachers of younger chil: 
were sambled from schools feeding into those selected at the 6th gra 



approximately 15 teachers in grades 7-9, -drawn randomly from the par 
ticipating^teachers in each school from which the 9th grade participants 
were selected, with a minimum of three teachers from each school*; and 

approximately 15 teachers in grades 10 through 12, drawn randomly from 
the participating teachers in each school from which the 12th grade part^ici- 
pants were selected, with a minimum of three tecfchers from each school. 



^Table 6 presents the number of pa^rticipatin^ teachers that were surveyed. As 
'indicated, approximately 14 % of the teachers identified as participating in the 
Part D projects responded. Responses to" the survey were obtai|ied in 49 of th6 ■ 
50 projects visited; the mean number of responding teachers per project was 29. 
More specificc^lly, the number of , respondents per project ranged from a low of 
six teachers to a high of 4§ teachers. The distribution was fairly even: in nine 
projects the nlimber of respondents ranged from 10 to 19; in l3 it ranged from 
20 to 29; ih 15 it ranged from 30 to 39; and in 11 it ranged-from 40 to 48. . 
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* This characteristic was measured by the observation of the test /questionnaire administrator, who assigned one of five ' 
codes to each ttudent< The code categories were Blade, Spanish-sumamed, American Indian, White and Other. The 
chi-quare tests were computed on the, basis of all five categories. " 
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) ' TABLE 6 ^ . 
i SAMPLE OF PARTICIPATING TEACHERS * 


V « 

Grade Level 


Total partici- 
pa^ting teachers 


Number' of 
respondents 


% of total 
responding 


Elementary''^ (1-6) 


4, 861 


586 


12. 1% 


Junior High'" (7-9) 


3, 325 


358 


10.8% 


Senior High-i^ (10-12) 


1,823 


489 


26. 8% 


Total \^ 


10,009 


' 1,433 


14.3% 

(average) 


^ Uliile teachere often had responsibilities cutting across categories, for sampling purposes they were assigned to a 
stratum on tiie basis of a judgment of project staff with respect to their major time commitment. 



The sample of non-participating teachers in each project consisted of the teacher(s) 
of the 6th ^ade non-participating students. Non-participating teacher data were 
obtained in 41 of the 45 projects in which 6th grade non-participating ^tudents were 
tested. The number of non-participants' surveyed per project ranged from a low 
of one teacher in ten cases to a high of nine tea^chers in one project . 
The total numbet surveyed was 122, and the mean number surveyed from the 41 
projects was 3. 0. 

The sample of guidance counselors consisted of all counselors defined as project 
participants. Counselor survey data were obtained in 44 of the 50 projects visited. 
The numbler of counsej^s surveyed per project ranged from a low of one in eight 
projects to a high oifz? in one project, 19 in another, and 13 in a third. 
The total number surveyed was 229, and the mean number surveyed from the 44 
projects was 5. 2. 

E, Instrumentation 

During the first months of the study, each of the eight legislative objectives was 
Maborated into a set of*testable hypbtheses. An initial set cJf hypotheses was 
developed by the Development Associates study team and then reviewed and mod- 
ified as appropriate by the USOE project officer and USOE Part D program staff. 
From this list, a final^set of hypotheses and their indicators \yas adopted based on ' 
the results of the pre-test, site visits and a consensus of USOE and Development • 
Associates staff regarding their applicability for each Part D objective. As the 
hypotheses were presented in the context of Chapter III and will be presented along 
with their indicators and related student outcome data in Chapter VIII, they will 
not be repeated here. * ^ , 

It was found that several of the student outco;ne variables contained in the hypoth- 
eses were adequately measured by one or more-^of the sub-scales of the Career 
Maturity Inve,ntory (CMI). '--'^ During the planning .stage of the study^ this test was 
judged to.be the most appropriate stan^lardized inSti^ument available, since, in 
addition to covering major variables of interest, it also covered grades 6 through 
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12, had a suitable administration t;me, and had been otandardized on students from 
throughout the United States, The Inventory consists of an Attitude Scale and a 
five-part Competence Test, These are: • ' 

Part 1: Knowing Yourself (feelf -Appraisal) 

Part 2: Knowing About Jobs (Occupatibnal Information) 

Part 3: Choosing a Job (Goal Selection) 

Part 4: Looking Ahead (Planning) / ♦ 

Parts: What Should They Do? (Problem Solving) 

The Attitude Scale is made up of 50 true /false items; each of the parts of the 
Competence Test consists of 20 multiple choice items. 

For variables not covered by the CMI, questionnaire items were written for 
studTents', teachers*, and guidance counselors' questionnaires, as appropriate.'*^ 
Separate student questionnaires were prepared for each of the three grade levels , 
being tested. Insofar as possible,^ for those items which were utilized at more - 
than one^rade level/the format and the wording of the items were the same in the 
thr^^mG^tionnaiYes, Questionnaires were also prepared for the teachers and 
guidSW^ggunselors , and all items and format were reviewejd by project staff, 
teachersfcounselors , and students during five visits to projects to pre -test mater- 
ials and p rocedures. These visits were scheduled such that modifications in items 
and formats resulting from the first visits We re ^.reviewed by individuals in projects 
visited later. The items and procedures were developed and modified so that total 
testing time per student would be under 2\ hours. 

^ addition to the questionnaire s^ which were completed by students, teachers, and 
counselors, a "project information record" (PIR) was developed for the use of the 
field stafi? The PIR contained a set of tables on which information gathered from 
.project and school files was recorded (e. g, , enrollment data, placement data, 
expenditure data,, etc. ), and a set of questions to be asked in the course of the on- 
site interviews. The interview questions and data tables were pre-tested during 
the five site visits made during the design phase of "the study. 

The initial study plan called for a survey of employers in each project S-ite and a ^ 
telephone and mail survey^f school districts in each state not receiving Part D / 
funds but reported as implerr^enting activities as a result of the funded projects. 
Questionnaires were developed for these two facets of the study,, but based on the 
results of the pre -test visits it was decided by USOE and the study team to elimi- 
nate these two aspects of the study since reliable data across prjects could not be 
secured with the available time anci resources. 

The three student questionnaires, the two teacher questionnaires, the guidance 
counselor questionnaire, and the project information record are on. file with the 
U.S. Office of. Education. 

\ 

F. Field Procedures » 

The various data collection instruments and the procedures for using them were 
field tested at five project sites, and revisions were made to reduce ambiguities * 
a.n^ to simplify the data-collection process. The field tests were conducted by two- 
person teams who administered the questionnaires and tested procedure s on a 
small numbej; of individuals in each site. Tests and questionnaires were not admin- 
istered during the study visit to tl^ose individuals involved in the field tesrts. 
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* The imtmmentation did not include tests of specific Job skills, since neither the legislative objectives nor the infonna- 
tion^obtained about the projects indicated that these were pertinent. The projects were not required to provide vocational 
training and jSo project placed particular emphads on such efforts. , 
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The data -collcLt ion plan called for a tvvo-person stu<iy team to visit each project 
for a period of one week. Each site visit was. first coordinated with the SEA and 
the dates of the visit confirmed by phone. At this time the information obtained 
in the DVTE mail survey mentioned^earliex was verified, and sampling and logisti- 
cal issues were discussed. '*lr 

Each project was asked to assist in the identification of suitable individuals to be 
employed a^ local testers when the team arrived on site. These testers were, 
drawn from the substitute teacher rosters and hired b>Q the on-site teams. The 
Igcally hired testers were then trained, on-site, t6 insure that the tests were ' 
administered properly and that the,y were' thoroughly familiar with the s ite -specific 
questions to be asked of each group of students. The entire procedure was then . 
rehearsed and each tester wa,s supplied with a roster of the stll^rits selected by ] 
the DA staff for testing, the time^and location of the testing, the name of the school 
contact person, and the tests and questionnai^-es . / 

The test administrations were spot checked by a field team member during the^ 
course of the site visit and each local tester \yas debrit^fed upon the completion of 
Ocich test administration, W ritten'notes on the administrations were rnade for use 
in the analysis stage. 

As an aid to the field study teams, a detailed field procedures manual was preparec 
This served both as the basic document us^ed in an intensive three -day training 
program for team members prior to theif first visit and subsequently during theii:, 
visits^ to projects. 

The field work was accomplished between mdd- February and the end of May, 1974. 
,Four field teams Visited eleven Jprojects each and a fifth team, consisting of the 
•study director and assistant director, visited six sites. The rnembership of the 
field te^anhs was consistent througout the study ^nd each team communicated vC'ith 
the stuAr dri*t>ctpr or assistant ^t least once a week. 



At. the c^pffpletion of each visit the teams reviewed all materials and packaged them 
for central data processing in Washington. Each team also completed a narrative 
proj'ect description and a narrative report sun^marizing their field procedures. 
Any deviations from the prescribed plans necessitated by local conditions a^nd any 
other information which might affect student outcome data was noted in 'iAiese 
reports and was utiUzed in data analysis and interpretation. The na rotative project 
descriptions, and tfte fiscal and participant data reported in Chapters y and VI, 
were sent to each jiroject for review and cdmmenfc. The narrative reports a^e ' 
compiled in a separate volume on file with'USOE and available through ERIC. 

G« Data Processing and Analy s is 
^ 

Student test data and all questionnaires were prepared for computer analysis^ For 
the test data, an optical scanning technique was used to put each recbrd on tape, 
followed by computer scoring of each record and merging each student's scores 
with his questionnaire responses. For the ques tionnaires , -^the process involved 
coding several variables , followed by key* punching and making corrections of 
errors found by means of computerized editing. 
% 
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The basic data were tabulated and analyzed usii^g the/formats and statistics of the 
computer program SPSS {Statistical Pachcage for the Social Sciences). The princi- 
pal format used for the computer outputs involving frequencies and comparisons 
of student data was participants vs non-participants withm grade level or 12th grade 
group. This format was utilized for obtaining both project-by-project data and I 
.totals across projects. For participating teacher and counselor data, tHe tabula- 
tions were^un on a within-project basis, and totals. by school level (elementary, 
middle, and senior high). Totals for 6th gra^d^ participating teachers and non- 
participating teachers were also obtained. \ ' 

The. overall questions which the analysis was designed to answer were: a) did 
participants demonstrate higher performance ^ith regard to pre -hypothesized 
"outcome" variables, and b) if so, are these differences attributable to program 
treatments, or othe r program. characte ristics ? • 
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CHAPTER V: FEDERAL INPUTS 
"A Comparison of the Planned and Actual Federal Inputs' 



Introduction 



Inputs are understood to be the resources applied to the attainment of an established 
goal or goals. The identification of planned inputs, of actual inputs, and a compari- 
son of the two are important to any evaluative study, 

^ 

'For purposes, of this Revaluation, it was de,cided to define the planned fede^ral inputs 
as the sum of the ftinds 'allocated to the 50 projects being studied as reflected in the 
grant actions of the USOE. The grant actions represent the commitment of funds. 
Therefore, this comm^itment does, in a very real sense, represent the planned in- 

•puts at the federal level. 

Iffthe grant actions represent the planned commitments of resources, expenditures 
must. represent the actual commitment' of resources . In determining the actual inputs for 
this study, itwas necessary to obtain the expenditure data for eachpioject. The ^um of thisy 
data.represents, for purposes of this exercise, the actual inputs into the*system. Table 7 
compares the planned inputs to the actual inputsrthat is, displayed are the d^ta coicerning 
USOE grantactions, projec t expenditures, and the difference bet^en these Jn perc entages ). 

The original study design called for this information to be obtained early in the study 
from the files* of the -ySOE. The grant action information was taken from these 
federal records. Expenditure data, however, was not available at the federal level 
and it was necessary to include this data in the information to be gathered on-site 
by the stud^ teams. ' 1 % 

• Budget and expenditure information was not generally available from project staff, 
inmost cases/ the LEA's fiscal office maintained the financial records. As the 
table indicates, six projects were una^ble to provide the information sought. This 
was due, primarily, to a conflict of demands the fiscal offices and the low prior- 
ity placeH upon searching three-yea3>old records in the press otother immediate 
needs. 



A review oCthe information available does, however, present a good picture of the 
relationship between the planned and actual federal inputs, ' ^ 

^ 

B, Findings - , 

« , ; * 

The table. indicates for each project the grant action, tlie project budget, and the ' 
.percent change between the two for^each o{ three years of federal funding, In^ddi-* 
tion, project total expenditure for the year and the percent change from budget to 
expenditure are indicated. 

An inspection of the' data x^eveals that 88, 6% of th^ projects reporting underspent 
their first year budgets. ^This represents 39 of the 44 projects reporting data for 
that year. The underspending ranged from a -0, 6% to -60. 8%. T^e average under-' 
expenditure was -19. 2% per project. The District of Columbia and Florida projects 
corfibined 142(d) funds and 142(c) funds. They were unable to provide information on' 
14^(c):»funds only. Hence, they reflect budgets in excess of the grant actions for the 
first year. 

The legisla^tion provides that "funds available to the Commissioner pursuant to sec- 
tion 142(c) shall remain available until expended. " Based upon this provision, ' 
unexpended project funds were usually carried over as an addition io the second 
program year budget except in the case of one project, where the program year was 
extended. This rebudgeting i« reflected in column "h" as the percent change between 
fhe second year grtat action and the second year budget (columns f and g). 

^ . ' • 35 - 
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Of the 4l5'projects fo^Avhich second year budget totals were reported, 25 indicated 
budgets higher than^the second year grant action. This represents 55, 6% of the 
projects reporting. ,The projects ranged fronn budgeting 3, 8% over the grant action 
to 71., 6% over with .the median being 14. 8% per project. Both Florida and District 
of Columbia have b^6n excluded from these figures because the reported data did 
n,6t permit separatii>n of the 142(c) funds Irom others. 

There we^re 14 projects that unde^pent their budgets the first year and did not carry 
over the funds the second year. W^ile no specific reasons for this phenomenon were 
discovered by the field teams, it appears that the nature of the budgetary prpcess, 
size of) the liEA, and mechanization of accounting functions were factors in the fail- 
ure o£ some of these LEAs tp rebudget first year carryover funds. In some instances 
the budgetary cycle was initi^ated too early for the fiscal sections to project carryover, 
request budget modification i)f OE, secure approval, and submit this to the school 
boa/rd. 

If^ larger LEAs the data processing center typically was unawa||| that funds could be 
rep rog rammed. They did not alter the normal fiscal program llhich did not provide 
^or such a contingency. ' ^ , * ' 

TVenty projects did not have an increase of fheir sedond-year budget over the grant 
acrion. In the case of four projects, this was due to their first-year expenditures 
being equal or nearly equal to the grant action figure. Two projects reported a 
second year budget that was actually lower than the grant action. 

The second year expenditures, like the first year's, were for the most ^art below 
the amounts .budgeted. Of the 44 projects reporting data, 86. 4% were underspent 
during the second program year. This indicates that 38 underspent during the 
second year. The average amount of the underexpenditure also was close to that of 
the first year and stood at -15. 96% (-19. 2% for year 1). The range of the second 
year underexpenditure ran from a low of -0, 7*7^ to -69. 5%. 
7 ' 

As can be seen, the overall second year data relating expenditure to budget is not 
basically different than that for the first year in the aggregate. Two projects were 
underspent the fi'rst year but not in the second yeaCr. One, in fact, slightly over- . 
spent its budget (+0. 6%). One project, on the othpr hand, did not underspend 
during the first year but did do so during the second year. 

Comparing the* second year expenditues to the second (year grant actions presents a 
^abstantially different picture, however. This c Camparis on reveals that the second 
^year expenditures jnuch more closely approximated tlte grant action figure than they 
did the budget figure. The projects uncierspent the second year grant actions by an 
average of -6. 30% as compared to the 4l5. 96% reported above. Thirteen projects 
reported expenditures that were higherWtiian the second year grant action buf less 
than their budget figures. . ^ ^ 

Twenty-six projects carried unspent second year funds over to the third program 
year. o There were 12 projects which haci unexpended second year funds that did not 
show an increase in budget over the third year grant action, indicating that the 
fvinds were not carried over. Ten of t^iese 12 we.re projects that did not carry over 
unexpended first year fvinds to the second year. 

One project carried first year vinderexpenditures over to thte second year and was 
not underspent that year, but reported a budget 8. 9% higher than the grant action 
for the third ye^^r. ^ 

Examination of the third year expenditures reveals that 26 projects underspent their 
budget put of 37 reporting expenditure data for that year. This represents 70. 3% 
ol the reporting projects. California did not report third year data because the 
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project was in its second yeatr of operation. In the otljer cases, the final budget and 
expenditure data was not available because the prej^cts h^ either not yet run the 
full third year term or had done so only recently and were awaiting final close out of 
their books. 

Underexpenditures ranged from a low of -0. 2% of budget to a high of -51. 8%. The 
average underexpenditure was 12. L%. * 

In the final program year, five projects reported expenditures in excess of the 
amounts budgete'd. The actual dollar amounts involved ranged from a low of $142 
to a high of $28, 000. With one except!^, these projects are ones that did not 
adjust their budgets to include carryover funds from previous years. During the 
first two program years only one project reported exceeding its budget. That one 
reported a 0. 6% over expenditure during the second program year. 

When considering the allocation of resources or the level of inputs into a system, 
time is an important consideration. If, for. example, the total allocation of resour- 
ces for a particular effort was to be three perason years, different levels of effort 
are de-scribed by three persons each working for a one-year period, as opposed to 
one person working for three years. While it was expected, based on legislation, 
that the Part D programs would be limited to no more than three years, it was 
found that 22 of the 50 projects studied had received time extensions and were in 
operation for more than a three -year period. These extensions in time were at "no 
additional cost to the government. Programmatically, this means that in 44% of 
the projects the expenditure of federa^^e sources was at a rate below that planned. 

C. Summary and Con^\asions 

In summary, .a review of the year by year input data reveals that in most projects 
in no single year did the inputs occu^r at the level planned. Taking the three years 
in the aggregate and considering both expenditures and time frames, it was found 
that in 37 (74%) out of the 50 projects the acttial inputs were below the level of the 
planned inputs. \ 

\ 

The generally high level of project underexpenditure in t^ieir first program year 
would appear to be attributable to program start-up requirements. * 

Several possible explanations for this apparently slow start-up suggest themselves. 
The first or>e relates to the timing of the grant awards. LEAs may Kave had diffi- 
culty responding to new efforts initiated after the school year had actually begun. 
The recruitment of certified personnel, for example, is increasingly difficult as 
the summer^nds. Most have signed contracts by early to middle August. Once the 
school yedr^has b^gun and schedules established, it is difficult to adjust the already 
full schedules of personnel to permit the assignment of new op additional! responsi- 
bilities artd .tasks. Table 1 in Chapter II indicates that 25 of the projeots\studied 
were awkrd^d their first year grant during the month of June, three in Jujy, three 
in Adgust, three in September, four in October,'' one each in November and Decembe 
four each in January and February, and two in March. 

Of the seven states reporting the highest rates of unde;rexpenditure in their first 
program year, three had grants av;ardeji in June^ two had awards made in the mid- 
dle of the school ye^ar (January and February) and the two r.emaining received 
awards in August and September.^ This does not appear to lend support to the 
hypothes.is that the timing of the grant may affect the time required for pjiiogram 
start-up: * ■ , 



* Prior; to using average figures in this section, both means and medians were determined and it was found that the use 
r>( one\ss oppofed to the other did not substantially alter the picture presented. The mean figures aw; reported. 

** Sec. J4S. "Financial assistance may not be given under this part to any program or project for a period exceeding 
threrf^ years. " 

i. ^ , ' 
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Another possible explanation for the speed with which the programs became fully 
operational relates to the nature of the initial project activities. As indicated in 
the next chapter, many reported that they emphasized planning, curriculum develop 
ment; and staff training and development during the first year. These activities; 
are generally not as expensive as are activities intended to deliver the plai^t^d 
interventions. The second and third year e:4?®ri^i^^^ss approximate the grant a^ltion' 
ci^nounts. This suggests that once underway /the projects adhered to their original 
plan. «^ 

The relationship between the second and third yedr grant action to expenditure and 
budget to expenditure figures clearly suggest that tn the secoijd and third program 
years the projects generally spent as originally planned. The projects carrying 
over unexpended funds from one year to another ^pear not to have spent these carry 
over funds. This indicates that typically the third, year vinderexpenditure is a 
reflectiou of the first year undeifexpenditure. This may indicate that the projects 
did not fully understand the nature of the carryover process and failed to adjust 
rheir plans to reflect the additional funds. On the other hand, it may also indicate 
that the projects planned to end the third program year with excess funds. In some 
cases this might permit the programs to continue to the end of a school year. In 
others, it may have been to cushion the effect of the termination of federal support 
and reliance upon local funding. 

Logically, it may be expected that thp absolute level and the rate of the expenditure 
of federal funds would have an effect on program outcomes, and the effect of rate 
and level of expenditure will be explored in the chapters which follow. 
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CHAPTER VI: FEDERAL ACTIVITY OBJECTIVES 



' *A. Introduction 

In Chapter 3, the planned federal activity objectives were defined as those atiL\ities 
outlined in'Section 143(a)2 of the legislation. These activities were expected to 
produce certain student outcomes, and the production of these outcoraes was to be 
considered an indicator that the Part D effort had contributed to the legislative goal 
of reducing the level of youth unemployment. Thus, an adequate explanation of why 
the student 'outcomes did or did not occur as anticipated requires an understanding 
of how successful the federal Part D effort was m reaching the planned activity 
objectives. ~ . . ^ 

As indicated uii Table 8 (p. 42) the federal legislation listed eight activities to be 
supported by Part D funds. The policy paper (AVL-V70-1), on the other hand, 
established certain priorities by requiring that all projects direct some level of 
activity toward occupational orientation, wo rk expe rience programs, intensive 
occupational guidance and counselmgi and the initial placement of all students at 
the completion of their schooling'. 

In addition, the policy paper specified that each project was to include an effort to 
broaden and improve the vocational education curricula by emphasizing activities 
designed to provide . . special training m job entry sliiils . . . for students not 
prev-iously enrolled m vocational programs . . . just prior to the time that they 
leave school. 

The remainihg legislatively defined activities were not addressed in the policy 
paper. A review of the oroject proposals and third party evaluations revealed that 
none of the fifty projects studied had program objectives explicitly designed to 
initiate the legislative activities . not covered in the policy paper and some projects 
nad objectives not included in the legislation. Therefore, this study focused on the 
policy paper activities, and included an*"other" category for projects with special 
activities. - 

In order to ass'ess the student outcomes resulting from activities presented in the 
policy paper it was necessary to determine which students would potentially be 
affected by each activity. The results of this matching of students and policy'paper 
activities is shown m Table 8. For purposes of this study elementary school 
activities were defined to include grades 1 through 6, junior high activities included 
grades 7 through 9 and senior high activities included grades 10 through 12. 
• 

According to the USOE policy paper, the national focus for the firs^ round programs 
would be on projects which combined in one operational setting each of the five 
required activities. The findings with regard to project activities' reported below 
first address the issue of the extent to which these activities were produced m the 
fifty projects studied. This discussion is then followed by findings which relate to 
the issue of the level of activity as indicated by the number of participating students 
in each project. It should be noted that the focus of this chapter is on the broad • 
categories of program activities.. Findings with respect to the more specific student 
treatments associated with each broad category will be presented in Chapters VIII 
and DC. 
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TABLX 8 PROGRAM ACTiymES 


Fedcr^ L<giaUticn Activiti«a 


Related Polic{r Paper Activities 


Student Grade Level 


• Faj;}UL2rue elemezitiry *Ad aecocdMy 
sM&Rt< with broad r&ag« of occupation* 

• Provide atud«ftt< with education^ 
experiences thrcvghwork 

• Provide mlcoAive occupatio&Al gMirtance 
, coucjelisg in last acbool years 

• Placement 

• Broaden or iMprove voca:;oiui edu* 
caticn curnculun 

• Produce p«rtonr.el exchanges between 
schools and ocher agencies and 
businesses 

• Provide educati^Wl opportunities for- * 
ou:^o^»school youth released part 
:une"fron their jobs 

• Stotivaie A^d prepare sr^dentj :o 
seex zraLT.u'.g aa vocaiio&al educarors 

• Other act-.vi:ics wbich pursue " ''^^ 
legialaiive objecrtvea 


Broad occupai^fnai orientatica 
at elementary and secondary 
levels 

Work experience and coopera^ 
ti'^e education programs 

In^nsive occupational' guidance 
ard counseling duringShc laat 
ytar* of schocf 

Initial placement o^ aU atud«nta 
at the compUcion of their 
schooling 

Jcb entry skill training for 
st>^ent« not previoualy enrolled 

N>< addressed 
No«a4dr«s«ed 
Not addressed 


Elementary, junior 
high, senior high 

Senior high 
Senior high 
Senior high 

ft 
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B. Findings 

* 

The findings related to project activities are presented in Table 9. The ta^ble 
reports the nunnber of participants reported by each project at the Elementary, 
junior high, arfd senior high school levels. The nuniber of participants for each 
of the three years of program funding is indicated. In addition, the third year 
participants are compared to the total LEA enrollment for that year. Finally, 
the table indicates for each project the senior high school activities specified in 
the policy paper for which some level of effort was found for at least one program 
year. 

The data for this table were derived from the responses of the grantees to the 
field teams, from grantee responses to a DVTE survey of participants in September 
1973; and from the grantee's interim reports and third party evaluations. Wher-c 
significant inconsistencies were found among these sources, the figures judged 
to be most cohsistent were used. 

i 

L Funded Activities 

All of the projects reported sonne elementary (1-6) and junior high (7-9) school 
participants for at least one year except the projett in Massachusetts. Twelve 
projects reputed no elementary school participants the first year. Of these, 
12, Utah reported elementary school participants in only the third year. At 
the^ junior high school level 15 projects reported no participants during the 
first year. Four of these did not have participants the second year, 
but all but one did have' by the third project year. Ten of the 11 reporting 
no elementary school participants for the first program year also reported no 
junior high school participants for that year. * ' / 
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At the senior high school level, 26 projects (52%) reported work experience 
participants during at least one of the three project years. Twenty-three 
lJ>rojects (46%) reported participants in job entryskill training . Thirty-one of 
the pr6jects (62%) reported high School participants receiving occupational 
orientation through curriculum infusion techniques applied by participating 
teachers. Twenty-tv^'O projects (44%V'reported*some high schobl participants 
receiving intensive counseling arfd guidance . 

T'hirty-seven (74%) of the grantees reported some' level of high school p'lace- 
ment activity supported in whole or in part by the Part D project. In general, 
projects (did not maintain records- which would permit them to report the actual 
numbers of students placed. Most often placements were recorded m each 
student's counseling file making retrieval difficult and tirne consuming. , In 
addition, the placement figures reported frequently represented an estimate 
based upon partial return of mail survey forms. Because of the difficulty en- 
countered in securing reliable placement information, placement data are not 
included in this report. , - • . , 



These findings reveal variation in the reaching of the planned federal activity 
")bjectj * ' ' ... - 



objectives. This variation is summarized in Table 10,, below 



TABLE 10. 




NU>lfeER OF pI^OJECTS REPORTING FEDERAL ACTIVITY COMPONENTS 




Number of Projects 




Activities 


Reporting 


% of 50 




„ Participants 


Projects 


Elementary Activities 


49 


98% 


Junior High Activities 


49 ^ - 


98%. 


Senior High Activities 




• Placement 


37 . 


74% 


♦ Senior. High Participating Teacher 


31 


^2% , 


• Work Experience Programs 


26 


52% 


• Job Entry Sk^l Training , 


23 ✓ ' 


46% 


• Intensive Guidance and Counseling 


22 


44% 


• Other Activities 


7 


14% 


—A ^ 





\ 

The policy paper required each project to combine . . in one operational 
setting, " all of the following' activities : 

• work experience programs; 

• job entry skill training; / ' . 

• intensive guidance and counseling; * | 

^ i 

• initial placement of all students; and ^ 

• occupational orier^tidn*at the elementary and secondary school levels. 



- 45 - 



The information'reported by the projects reveals that 13 projects (26%) parried 
out all of the activities as required in the policy paper. Thus, in 13 projects 
the planned federal activity objectives were reached and in 37 they were not. 

Level of Activities 



The policy paper requirements did not indicate thedevel of effort expected of 
any activity other than placement (which specified all students comj>letinig 
schooljT. Level of effort is also not addressed in the legislation cr the Federal 
Register. It is, however, an important consideration in*ariy evaluation. ^ Table 
9 identifies the activities of each project and indicates overall level of effort 
in terms of the number of participants at the elementary, junior high, and ^ 
senior high school levels. 

The fifty projects reported a Jotal of 231, 176 third year participants. Of that 
number, 120, 584 (52. 2%) were elementary level (gradea^^sivT^) students, cind 
77, 348 (33. 5%) were junior high level istudents (grades 7-^). The remaining 
33,244 (14. 4%)*were high school students. At the high school level, 2,974 
" work experience students were reported* and there were 6, 642 skill 

training participants, 18,604 participating teacher students, 3,813 counseling 
and guidance participants, and 1,211 reported in the "other" project activity 
category. / ' 

These data indicate tliat 55.9% of the hi|h school level participants were invol- ^ 
ved in occupational famili^kVization activities defined as infusion of career 
education concepts int<j the regular class room curricula by participating teachers. 
Job entry skill training activities involved 20. 0?i- of the high school participants. • 
Counseling and guidance activities involved 11. 5% of the total while work exper- 
ieffce accounted^or 8.9%. ' * * - 

Comparing the number tiiird year participants to the total school enrollment 
in the, third year revea'ls wide variation between projects. Eleven projects * ^ , 
reported their entire elementary enrollment as project participant^^ Two 
projects, on the other, hand, reported fewer than one percent of their enroll- 
ment as 'project participants. 'Six projects reported 100*^0 of the junior high 
enrollment as participarft? ?,nd seven projects reported all senior high students'* 
|as project participants. Four projects reported the entire LEA enrollment as 
participants; Arkansas, Idaho, New Mexico, and Wyoming. , ^ 

It was found that many projects had difficulty identifying which students were 
•actually p*roject participants. This difficulty was reflected by their inconsistent 
respo;ises concerning the number of participants. Comparing the.^data provided 
the field teams during the^si-t^^isits and the data provided by the DVTE survey, 
mentioned earjlief in this chapter, ' revealed figures that varied as much as 84% 
for one project. 

, Projects appeared to have the most difficulty identifying senior high school 
' leve^ participants in familiarization activities. This was largely because the 
-.focus of fnost projects was on the participating teacher. Since a given teacher 
wodld have as many as five different classes of students, tracking byi student 
* was usually top difficult and time consuming. A similar pattern emerged when 
intensive guidance and counseling activities were carried out as separate activ- 
ities* rather than as p^rt of a skill training (^^^ork ex^rience component. 



In a separate sjudy task, the grantees were asked to provide information concerning the <:ost of each area of project 
activity, The responses produced virtually the same picture of project activities as "did the identification of activities 
and participants discussed above^ With only 37 grantees responding, the number that reported costs in all five of the 
minimum activities wa$*l 6. (See Table ^> Chapter X). * - , 
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Becausowpf heavy case loads and because participating counselors usually 
engaged in many types of counseling, it was idffi:cult for projects to identify 
spec«c students who received occupational guidance and counseling. 

From the tat>le it can , be determined that projects averaged in the third program 
year (as determined by the maan) 2, 512 elementary school participants, and 
that these participants compnsed on the *ave rage, 45. 5% of the total elementary 
school enrollment of the grantees. For the junior high school level there was 
an average of 1632 participants per project, comprising 46. 9% of the total en- 
rollment. At the high school level the overall number of participants averaged 
a total of 723 students per project. This indicates thAt th^Tbtal project effort 
at that level involved an average of 34. 7% of the high school t-nroUment for the 
project grantees. For each high school activity group, the average number of 
participants in projects having the ^.ctivity was reported to be: 

e work experience - 128 students pf^f project; . ' * 

• skill training - 3 50 students per project; 

« participating teacher - ^42 students per project;/ 

• » counseling and guidance - 484 students per project; and 

• other activities - 242 students per project. ' 

Table 9 ^Iso indicates the number of participants for each of the three years 
of project operation. These data indicate that the projects tende^ to increase 
the num^r of students affected each program year* *i In general, even the 
projects ^reporting a decrease in the number of participants for a given group 
from year tp year indicated that the decrease r-epresented a decrease in the 
enrollment in. the participating schools for that year, as opposed to a planned 
reduction in the number of participants. Increases, on the other hand, tended 
to reflect the addition of new schools tcrwhich familiarization activities had 
been extjsnded or an increase in the number of teachers involved in the project. 

C. Conclusions 



In Chapter V it was concluded that the federal inputs did not occur as planned 
across all 50 projects. As indicated, there was an average project under expendi- 
ture of 19. 2% of the approved budget for the first project year, 15* 9% the second 
year, and 12. ifo the third year. Further, giv^en these findings, it was ^expected 
that the^verall level of activities would be somewhat less than planned. 

. •. . ; — 

The data presented Ixi Table 9 and discussed above indicate that both on an annual 
basis and across the three years of program operations the e?ctent to which the 
activTties specified in the policy paper were carried out varied considerat9.y across 
the 50 projects. In ts'rms of the three year totals, while most projects reported 
students ass ociated w ith most of the specified activities, relatively^ few {26%) 
reported students in all of thenji. In addition, among those reporting students, the 
number involved varied considerably from project to project and over time. 

In general there was a marked increase in project activity in the second year pf 
operations. For*example, ten projects reported no first year elementary level 
parKk:ipants but did report such participants for the second year. At the junior 
High school levels eleven projects reported the same experience. At the senior 
high school, in seventeen cases there w^ere participants^ in an activity group the 
second year but not during the first. By the same token, in fourteen projects 
the rate of increase in the niimber of participants at tlie elementary grades was 
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greatest the second program year. At the junior high school level this occu;rred in 
eighteen projects,, and at the senior high school level in fift^erf projects this wa^ 
true for one or mo;re of the activity groups. 



This increase during the second year^TB^o ns igTent with the finding in Chapter V 

regard to the relationship between project budgets, expenditures, and gran^ 
actions. As indicated, during the second and third years most projects ex^^ended' 
Part D funds at a rate consistent with their budgets as originally prepared' and 
submitted with their'^rant application. These findings with respect to the increas- 
ing level of activities over time tend to confirm a conclusion suggested in Chapter 
V that during the first year many projects fopused on planniiig and other start-up 
activities which were relatively inexpensive and did not have an immediate impact 
on students. As indicated by the three year findings, however, it can not be 
assumed th^t the explanation for the cross project and cross -year variations is 
solcy accounte'd for by first year activities and project start-up requirements.^ In 
the chapter which follows a discussion of the relationship between project activities 
and the level of expenditures is presented. ' In addition it will explore the relation- 
ship between selected aspects of program management and the variatiohs across 
projects in the activities produced. 
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. CHAPTER VUr PROGRAM MANAGEMENT 

< * 

* , , . ' A. In^troductLon 

Chapters and VI discuss the federal inputs or resources p/ovided to the 50 
projects studied and the extent ^o which tl^ese inputs resulted in activkies as out- 
lined in the legislation, and required in the policy paper. From the diCcussion of 
inputs it was concluded that in most projects the inputs did not occur at the level 
planned; that is, in mo^t cases the projects did not expend all the federal funds 
Available to them. The unde rexpenditure s in the first program year typically may 
have been due to project s^art-up requirements; in many cases, the three-year 
total of unexpended federajl funds mdy be explained in terms of the amount not ex- 
pended during the first year of .operation. In some cases, however, projects 
reprogrammed unexpended fuhds and, according to their budget documents, planned 
to spend them the following year; in some of these cases the reprogrammed funds 
were expended as budgeted and in other cases they were not. 



On the basis of the findiix^S with respect to the rate and amount at \yhich federal 
P^-rt D monies were expended by the 50 projects, it was expected that variations in 
the level of project activities attributable to federal inputs woufd be found. The 
findings presented in Chapter VI with respect to project activities confirmed this 
^ expectatk^n. In fact, it was found that not only did projects vary considerably in 
terms of the»level at which activities occurred, they varied in-terms of the very * 
presence of activities the federal legislation and the USOE policy paper required 
of Part D projects. 

The Implicit assumption In both the Part D legislation and USOE policy Is that the 
funded activities at the local level are expected to produce student outcomes consis- 
tent with the purpose of the Part D program. Given that the inputs did not occur as 
planned. It Is logical to expect thpit the planned activity objectives would not be 
reached as, In.fact, they were not. This, In t^rn, leads to the reasonable predLc- 
tlon^that student outcomes would not be produced as planned. For this reason It Is 
appropriate at this point' to explore possible explanations for the failure of the ex-, 
pected activities to occur. In view of the findings discussed in Chapter VIII which 
support the general conclusion that In most projects the student outcomes of the 
Part D effort were not achieved, this explo ration regarding the relationship between 
program Inputs and the activities which resulted Is especially significant. 

Logically, the explanation for the variations across the 50 projects on the expendi- 
ture of federal funds provided and the activities which resulted may relate either to 
the nature of the federal management and funding procfesses^or to the management 
practices of the recipients of the.federal funds. Put another way, the first question 
whlch^must te addressed in explaining the relationship between federal Inputs and 
project activities Is: Did USOE actually fund projects which were expected to pro- 
duce the Part D activities called for by the legislation and policy paper? The 
second Is: Is thetre anything associated \vlth either the nature of the organizations 
receiving the grants or with the practices they followed in managing the federal 
funds which may exp^laln the activities which 'were produced? From the federal 
perspective both questions relate to the broad area of program management. 

The findings presented below are divided Into three categories. The first relates 
to the nature/ of the stated objectives of the projects which were funded. The second 
category rejates to the management practices followed by recipients of 
Part D^^ants. Specifically, these findings relate to the 

extent to which the* projects complied with management practices which were 
address(e»d by federal policy. Since these practices were required of all Part D 
grantees 'by virtue of receiving federal funds', the findings address both the 



questions of whether the grantees compiled with these requirements and whether 
compli^nceN-uas/associated with the nature of the activities produced. 

The thrrd catfegoru/relates broadly ta\he nature of the organizations receiving 
Part D grarnts. y/The fLndlngs here relate to the question: Does the nativt^of the 
grantee i\fe.,f/hether it is an SEA, LEA, etcv) relate* to either the nian^enient 
practice^ foU/^'ed ot- to the nature of the activities produced? In other words, 
does th^' natfiAlLaf^the organization i;eceivin^'Part D funds explain variations in 
activities not eAprfainable in terms of project objectives or specific management 
practices. THi| assessment may be of use in making program management 
det?isions ii)"yie future . - / ' i - 

' / ' * ' 

• / B'. Findings 

K PVoj^ct .Objectives > / 

" / . 

Prior to the field ph^se of'the study, the statements of project objectives, con- 
tained in project proposals, interim reports, and third party evaluations for 
each of the 50 first round projects in this' study were reviewed and recorded. 
During the. field vis its^project staff were aSked to confirm the accuracy of 
these statements or^ifpdate them as appropriate. The statements of project 
objectives were' then related to«the sev^h activity areas contained in the Part 
D legislation and, to the activity perfornlance- data reported in Chapter VI. A 
project objective Avas judged to^elate to legislative .objectives either if explicit 
reference was m4de or if in the combined judgment of the two senior project 
analy.kts who 'reviewed all project statepierits the legislative objective was 

^ cleanly implicit^' \ 

ivi,th; respect, to ."the prograrrf objectives contained in 'khe legislation but not in 
^he tJSOE poricTj paper, it was fourfd that-Adne of the 50 prdjects had 
sjta'ted objectives, Fjroviding for exchanges a£ {personnel betweeti schools and 
o^l^er agencies: - .Similarly, none of the project's ihdicated objectives which 
jVjO'ultd provide* jjicreasing the educational attainment of young workers 
l-el-eased fron^ thqip- jobs on a part-time b^jals. 'It was ajsa found that none s>i 
tli^e ^ 50 projeyfe Usl^t;ed the motivation of young people to b^come vocational 
e^u^c actors or tip pr^dvide pre -professional -training for potpVrtial vocatjt)nal 
edt;iijatprs as an oVj?ctive. Findings with respe<it to program obje'ctnres found 
in^i>i)t^i the poliqy.pfeijer and legislation are summarized lielow. 

^ jyiementaryYnd\>^econdary. Familiarization' - ' ^ 

F^rty-seveij af ^th^ fifty projects stjated objecfeJ^Ves wi^:^ respect to famil- 
iarizing elemeVifary and secondary school stu^^nts with the range of 
o^QjipVtions anc!»?|t4pis ites for those occupation^ . Of'the three remaining, 
found to*l\aVte actually engaged in elementary and secondary famil- 




elementaVy students (Jur^^g the final program year>* ; 



Work^ExdeVience' *\ ' vVV ' " . >* ' * 

; ■ ^ ..!• :,: 

Thirty-^'tK^e^.Droj'ec ts stateci^bjec.tives clearly relating to the provision 
^. \ of worl;' efj^^^Jehce activitio^S ^o hi^h school students arid ]7 did not state 
^/Isuch olpijlerctu^Wfe or did not ad^^ressr: tl4e subject clearly//. Of the 33 stating 
• \ {vork eipen^e-uc^ objectives, .♦J?.(57.6%) were found tbl.^'aye actualfyT: 
i-^jVjljI: *' _i * r. . .1., 



' 'Jrientec^iwiE^e^ferience^or <:::*a(^pprative education components at 
t.^C the proj&ts*;ari3 14 did notl» 'Of the 17 projects ncj(;itating woi 



ibutable^ 
rk"* 



experience objectives, seven were found to have irw^emented some level 
of work experience activities as part of the elementary program. As was 
indicated above, this would suggest vague project objectiv'5^. 

Job Entry Skill Training ^ 

Forty-one projects . indicated objectives clearly relating to broadening and 
improving vocational curricula. The policy paper indicated that activities 
designed to provide specific trailing in job entry skills for students not* 
previously-enrolled in vocational ptograms just prior to the time they 
leave school were to be included in each exemplary project. This priority, 
was established for each project and relates to the legislative objective of-^ 
broadening and improving the vocational curricula. Of the 41 projects ' ; 
stating such objectives, 22 were found ,to have initiated actiyities providing 
job entry skill training attributable to the exemplary projecf: and 19 di<i not. 
The nine projects not stating objectives related to skill training were I 
' found, with one exception, not to have implemented skill training . ' 
activities. , , 

t • ^ : 

Intensive Occupational Guidance and Counseling, and Placement ^ 

Forty-one projects indicated objectives relating to the provision of inten- 
sive occupational guidance and couns/eling to students during their last 
y^ars of school. Only 32 projects, however, also included the actual 
placement of students as a clear objective. Seven projects did not list . • 
either counseling or placement objectives, and two indicated placement • 
objectives but not counseling and guidance objectives. Of the 41 projects' 
stating counseling and guidance objectives, 17 were found to have imple- 
mented a counseling and guidance component identified apart from other 
program treatment and 24 did not. Three of the nine projects thajk did not 
state such objectives were found to have implemented such activities and 
six did not. 

Twe?ity-five oTthe thirty-two projects with stated objectives relative. to ^ 
the placement of students were found to have implemented such activities 
as a part of the exemplary projects, and 12 of the , 18 - projects 
not stating placemerft objectives were found to have actually carried out 
such activities as part of the project effort. . » 

Other Objectives , ^ ** 

In addition, 42 (84%) p rojects were found to.have objectives which related 
to activities not mentioned in the legislation, Ten of these projects in- 
cluded staff training objectives in this list of "other'^ objectives and six 
of these al^ included dissemincition objectives^ No other patterns ^i^re 
found in the statements of obj.^ctives. 

Thirty-two projects stated one or more objectives that were eitl^er too 
general or too unclear to be classified into one of the above categories, 
or in terms of additional pla,nned activities'i Forty-four projects stated 
objectives relative to some proposed activity other than the seven 
discussed here. ' ' , ' 

That projects would have stated objectives and implemented activities designed 
to'^fea<^*''*them is to be expected. On the othei' hand,, for projects not to have 
stated objectives addressing required areas or to have ste^ted objectives" bu 
no activities is n6,t. - • , ' * ' 



The major explanation for this phenomenon rdates to the prol5\em of project 
definition. The policy paper indicates special favor would be granted to 
projects incorporating funds from a variety of sources into a single project. 
At the time the grantees were preparing their proposals, many had already 
implemented ay/ork experience or 'cooperati.ve education effort within the 
LEA. The ongoing effort was in many instances cited as fulfillment of the 
work experience reqaifements . When no changes in the regular, pngoing 
effort attributable to the Part D project were found, the effort was not consid- 
ered for purposes of this study to be a component of the exemplary project. 

In three instances project perkpnnel and literature referred to jointly funded 
efforts which "were latpr determined %o be distinct'and separate administrative 
entities. In these casfes the Efforts also were not [Considered to be cpmpo- 
nents of the exemplary project. * 

It should be clearly pointed oufthat no willful mismanagement or misuse of , 
funds is implied by these findings, C)n the contrary, the phenomena discussed 
here appear to have resulted from unclear definition of the projects in terms 
of objectives, scope, and participants ^nd upon good faith efforts to maximize 
the effect of limited program reso\jrces by avoiding activity areas being 
addressed by other iijitiatives , theVeby narrowing and intensifying project 
focus. 

This, of course, implies that in many projects attempts to depart from the 
traditional offerings,, especially with respect to w-ork experience and skill 
training efforts, was limited. This may hav^ resulted from a scarcity of 
research and developmqnrtal efforts, in these areas or because grantees were 
not aware of the results of recent research. 



^^Jable 11, below^ summarizes the findings with respect to the relationship 
between project activities and objectives. 



^ TABIE 11; 

PROJECT ACTIVITIES AND PROJECT OBJECTIVES 


i 




^ of Proiects Enf^a^ed 


in Activities 


Specified 


n USOE Policy Paper 




Category ? 

1 


Elementary G Secondary 
Familiarization 


Work , 
Experience 


Skill 
Training 


Guidance & 
Counseling 


Placement 


Projects "with objective and activity ! 


47 


19 


22 ' 


17 


25 


^ : ! 


Projects with objective but without activity 


JO . 


14 




' 24 


-I 




Total with objective 


47 


33 ^ 


41 


41 


32 




|iBojects without objective but with activity 


; 2 


7 


. 1 


3 


12 


i ' 


Pipjects without objective and without activity 


J. 


10 


_8 ' 


A 


A 




' Total without objective 

, » .c 


3 


17 ' 


9 


9 


• 18 
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Project Management I^yactices ' 

This category of findings concerns the management practices followed by the 
individual projects. , As noted i^^preceding chapters, this study was not in- 
tended to be an evaluation of individual projects, nor was it intended to focus 
on project manage^Tnent. In part, this was because formal Part D effort had 
ceased in 29 casefe when the study teams visited the projects, and in many , of 
these cases key project personnel were not available for interview. As a | ^ 
result, t|oth the depth and quality of the information gathered across the 50 
projects jwith respect to project management varied. Nevertheless, it was ^ / 
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possible retrospectively to gather consistent and reliable infor^nation on 
several generic aspects of project managemenft which ipay help to explain the 
variations in project activities resulting from Part D grants. These findings 
are reported ih six subsections below: 



Sound Plan of Operation; 
Project Evaluations; 
Linkage to Is^anpower Ag-encies; 
Project Directors; 

Project Dissennination Activities; and 
Project Continuation and Replication. 



Each of the four subcategories on which findings are reported is based oh 
language found in the USOE policy paper, the legislation, and/or the Federal 
Regrster. As such, they may be considered as required of reci^ents of 
Part D grants . • 



a. Sound Plan of Operation 



The Federal Register (103. 25;a;Z) specifies that each project was to be 
judged, in part, upon the soundness of its proposed plan. It was further 
specified that included in the criteria for soundness was consideration of 
the extent to which the procedures for^ach'ieving the objective^ of the 
project were appropriate and technically sound. Several indicators of the 
soundness of each project's managerial operating procedures were col- 
lected. While no single indicator may be considered a sufficient measure, 
taken together they provide an indication of project operations. In addi- 
tion, although a positive response to the questions posed below does not 
signify, that a project was operationally sound, it may be argued that a 
negative response to more than one of the questions suggests that it was 
•not. "The results of reviewing project information in terms of these 
indicators of basic management practices are summarized belowi ^ 

i. V^ritten Work Plan 

In each site the study team was to determine whether or not the 
project had a written work plan detailing project activities in relation 
to time, and if so, whether the plan was used to guide and control 
day-to-day project operations. It is ^commonly held that such a plan 
is an invaluable tool in managing any endeavor. 

As indicated in Table 12, below, 31 (Sf the 50 projects were reported 
as having a written plan of work. Of those with such a plan, it was 
found that it was utilized as an operational guide to the project in 
58. 1% of these cases. Because the inability of the study team to 
identify work plans may have been directly related to the fact that the * 
projects were terminated (i.e.,.. no longer expending Part D funds) 
at the time of the visit, the data in Tcible 12 is divided in^to two cate- 
gories based on funding status. Jhese data shov that only of »-^c 
ongoing projects that had a plan utilized it as an operational gijide. 





TABLE 12 

OF'WRITTRN WORK PLAN GOVERNING P ROJ E C T. O PE R A T ION S 




Tvpc oi 
fVoicct 


Total ti of Projects 


No P!.'n 


Written Blan 


of total with plan 


^ Guided by plan 


,u with plan 
j^uidcd by plar 


(i^n^oing projcctr 
Terminated projects* 


29 


9 

10 


12 
19 


65 5 


6 
12 


50 0 
63 2 


Total 


50 


19 


31 


02 0 


18 


58 I 



J \^ ' Termin.ucd projects arc those not expemlhi^ Part D|c ) funds at^ic time of visit, those oixiratinj; on 'Vio cost" extensions 

arc considered onj^oinjj; 
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A comparison of the 18 projects which reported that they utilized a 
written work plan to guide project operations with the findings re- 
ported in Chapter ^VI revealed that eight p rojects had essentially 
performed federally expected activities which conformed to their 
• , ' stated objectives and ten had not. In general, there appears to be 
no meaningful relationship between utilizing a written project work 
plan and engaging in expected federal activities. 

ii. participant Data ^ 

As discussed in prior chapters, at various points in the study 
efforts were made to determine the numlJer of students participating 
in project activities. Also, as indicated, the projects varied with 
respect to the ease and consistency with which they could provide 
this information. The response to the DVTE mail survey seeking 
participant information by one of the projects summa rizes the 'diffi- 
culties encountered by many projects in maintaining accurate records 
of participants, especially at the senior high school level. 

"It becomes increasingly difficult to determine the number of 'participating' students 
35 they progress through junior and senior high school. In most junior high and in all 
high schools, students attend classes by subject and not by grade level. Therefore, 
even if we know that a certain Biology teacher is participating in [the project] it is 
very difficult a^d time consuming to determine. how many freshmen, sophomores, 
juniors, and seniors he has in that class." 

Nevertheless, given that the purpose of expending Part D funds was 
to produce activities which would affect students, it follows that 
sound^project management typically would require accurate and con- 
sistent records of participating students. Without such records it 
would at best be difficult to plan activi£f5es and to evaluate results. 

Operationally, 8ie indicator of the accuracy and* consistency of 
project records regarding student participants was the consistency 
of the data reported by the projects at various points in the study. It 
should -be noted that this does not directly address the accuracy^ of 
the records, but rafher the issues of 'their very existence and their 
internal cOnsisten<^y. It may be presumed that if the data reported 
is ^nob- consistent, the records are likely to be incorrect. 

I^was found that 15 projects reported data to the study, team which 
. was inconsistent with that reported in the DVTE participant survey. 
In one case there was a variation of 84% in the number of participants 
reported. In ahother four projects, the grantees were unable to 
report the number of participants by grade level to ejther DVTE'o'r 
the study team. a 

As with student records in general, it is reasonable to assume that • 
the management of a project having the placement of students com- 
pleting high school as a major objective would be aided by records of 
the numbers placed. Again, while to maintain' such records does not, 
imply sound management practices, the absence of such records does 
suggest 'managerial problems. ' . ^ 

\^ 

, Twenty-£hre^ (23) projects reported that records were maintained 
i*elative to the placertient of participating high school students. Two 
other projects indicated that placement records had been maintained 
but had'been lost. Twenty-one (21) projects indicated that such 
, records were Tit>t maintainejj. Nine of the twBnty-one projects 
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indicating that the project did not maintain placement records did 
indicate, however, that the grantee maintained such records inde- 
pendent of the project. Four projects did not involve high school 
students and theo'efore did not need to maintain records concerning 
high school placement. 

Wide variation was observed in the method by which placement 
records were generated, maintained, and used. Placement records 
per se were not generally available. In most areas placerre nt activity 
was entered into each student's coonselLng record. The retrieval 
of this information frequently involved a search of each student's 
record and the generation of aggregate data. 

Projects also varied considerably in their operational definition of 
the term "placement. " For some projects placement was understood 0, 
to mean the placement of a student an entry level job after graduation • 
or during the summer. For others itmea^it placement in a work 
experience position, and for most it meant both. In some cases, 
placement records did not distinguish between positions secured 
through the assistance of the school as opposed to the initiative of the 
students; in other cases, notations wer^ made of referrals to job 
openings rather than actual placement. These factors combined to 
produce data that were incomplete, vague, and incapable of compari- 



iii. Fiscal Control 

i^s indicated in previous discussion, each Part DVi^a^^g^^ received 
^ funds in order to perform activities which wQuld projj^ce student 

outcomes. It is generally accepted that an imporferfnt aspect of project 
management ia the monitoring of expenditures so a» to insure that 
^ resources are being expended at the level and rat^planned so that 
adjustments can be made where indicated and necessary. Clearly, 
in order to perform this critical management function it is necessary 
that project administrators have a record of expenditu're s and that 
'this record be in a form*yhich is useful from a progr.am management 
point of view. 

In the process of gathering data for the cost analysis of the Part D 
program (see Chaptef X), the study team reviewed expenditure 
records. In addition, they sought either documentation or reasonable i 
estimates from project staff which related expenditures to the major 
project activity categories. It was found that six projects were un- 
able to provide expenditure data to the study teams. In addition, 
three projects were able to provide only partial expenditure dat 
Twelve projects were unable to provide estimates of project expendi- 
tures by activity area**. These included seven of the projects report- 
ing partial or no expenditure data and five others. 

It was found that expenditure data was not usually available from 
project sources.' When seeking such information, the field teams 
were usualli^efer red to one or more administrative offices or to a 
grantee fiscal section removed from the project. PVoject directors 
usually did not include a review of timely e^^penditure information in 
their normal monitoring processes. Regular reports of project 
expenses typically were not required by grantee boards or personnel 
outside the normal fiscal processes of an LEA*. 

* 

- 55 - 



♦ 




r 



Sound management requires monitoring present effort, comparing 
that effort to the planned effort,, and making adjustments to correct 
any deficiencies. The fi^jrl^ngs in Chapters V and VI clearly imply 
that the management of project activities generally failed to make 
these kinds of adjustments and these findings with respect to fiscal' 
management suggest that Weakness in this area may provide at least 
part of the explanation. 

iv. Personnel Job Descriptions 



A commonly accepted indicator of. sound management practices is the 
presence of job descriptions for salaried personnel which specify 
duties and responsibilities associated with each position. In the 
course of the field visits the study teams reviewed the job descriptions 
of the staff paid from Part D funds. Thirty-two projects were found 
to have written job desQriptions and 18 either di<3 not o,r were unable 
to provide them for review. Several projects were found to have 
excellent job descriptions in which dulTies were clearly stated, report- 
ing lines defined, and objectives stated in specific terms. Many, 
however, did not have job descriptions written for the Part' D projects. 
Frequently, ^the job descriptions reviewed were found to be general in 
nature and applicable to all personnel within the school system holding 
a particular position, rather than specifically related to the Part D 
project. 

In summary, 19 of the 50 projects had favorable assessments with respect 
to three of the four indicators discussed above and seven projects were 
assessed favorably with respect to all four. 

b. Project Evaluations • 

It was stipulated in the Federal Register that each project was to be eval- 
uated. The USOE further stipulated that the evaluations were to be con- 
ducted annually l^,a third party and that a copy of the results of ^e evalu- 
ation was<«to be forwarded to USOE for review. The Part D projects were 
authorized to expend federal funds in order to employ a third party evalua- 
tor each year ctnd were expected to utilize the results of the evaluations to 
improve project operations. A final report at the end of three years was , 
also required, y- 

As discussed in previous chapters, the evaluation reports on file with 
USOE in the summer of 1973 were systematically reviewed priox to the 
start of the field effort. Reports for 45 of the 50 projects were available 
for review, and a summary of that review is presented in Table 13 on the 
following page, * • • 

Although the study teams were not responsible for reviewing the evalua-^ 
tions in each site, in the course of gathering fiscal data they did determine 
whether or not evaluations were performed each year. It was found that 
Part D funds were expended for third party evaluations in the last two 
years in all 50 of the projects. Since all projects complied with the re- 
quirements for third party evaluations, this factor does not explain the 
• variations in project at:tivities . 
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TABLE 13 

SUMMARY OF THIRD "PNRTY E \ A L U A T lO.NiS ON FILE WITH USOE 
DURING JUNE \\o lULV 1973 ' » 



Djte of 


Tester! or suncwd 
students 


tests 


Reviewed placement 
of :>tudents 


followed up 
s;r iduates 


Number 
ol 




Ye. 








\cs 


No 


Yes 


No 


projects 


1971 


1 


5 




6 


^- 


6 






6 




6 


1972 


9 


9 






2 


16 


2 . 




16 




IS 


1973 


13 


6 


t 


14 


' — """^^ 




2 




19, 




21 


TOTNL- 


25 


20 




32 


4 


41 


4 




41 




45 



Five projects did not have e>jlu[.ition reports on file Aith USOE, findings irom th^most current report are reported 



In the course of the visits the teams often reviewed evaki^EttOn reports 
which were not on file with USOE and discussed the utiifty of the evalua- 
tions with the project staff. Reports of these discussions suggest that 
the summary findings in Table 13 present a gene rally accu rate; picture of 
the evaluation eflort during roundy^ne. 
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From the discussions with project staff it was concluded tliat in most 
cases the projects viewed the evaluations as a requirement of USOE rather 
than as a tool which was 6f assistance in project management. This is 
consistent with the previous findings regarding project management. 
Apparently, many evaluations neither assessed student outcomes nor 
contributed noticably to a sound plan of operation. 



Linkage to Manpower Agencies 

The Part D legislation, the* Federal Register, and the USOE policy paper 
each indicates that projects should be designated to "...promote coopera- 
tion between public education and manpower agencies." The policy p^per 
further indicates that the required placement function of each project 
should be accomplished in cooperation with appropriate employment 
services or manpower agencies, in the course of the field visits the , 
study teams determined from project staff whether or not there were 
formal agreements between the projects and manpower agencies. 

Fourteen projects reported formal relationships with such manpower 

• agencies as the State Employment Service or the Vocational Rehabilitation 
Service. For the most part these relationships' were found to criginate 
from the mandated local manpower planning mechanisms emanating from 
provisions of the Comprehensive Employment and Training Act (CETA) 
^nd its predecessor, the Comprehensive Area Manpower Planning System 
(CAMPS). While all but one of the 14 projects reporting formal ties also 
reported placement activities, 21 projects without formal relationships to 
manpower agencies als*o engaged in placement activities. 

» 

In some of the projects not reporting formal I'elationships with. a local 

* manpower agency, the exis.tence of informal ties was indicated. While 
no systematic effort was m^e to investigate the nature of the' infarma- 
tional ties, typically these consisted of counselors contacting employment 
service offices regarding job openings for students. In this context, it is 
of interest to note that the guidance counselors surveyed were asked 
whether Cheywere on loaa to the project from another organization and, 

if so, from w-Hat type. Of the 229 responding counselors, only two were * 
on loan and those were not from either st*^t<? or local employment services. 

... .\ " 
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Proj.&ct Directors 

It may be presumed that by definition the directors of Part D projects had 
substantial impact .on the operations at the local level. In the course of 
the field visits the study teams obtained information indicating the extent 
of turnover during the life of the projects and salient background charac- ^ 
teristics of the raost recent project director. The background character- 
istic's gathered relate to the provisions of the legislation and the Feder'al 
Register which Indicated that project personnel should be qualifie'd by 
virtue of Unique and relevant experience. The Federal Register indicated 
th^l^nique and relevant experience In lieu of formal degrees and certifica- 
tion could be considered in qualifying project personnel. 

The following types of information regarding the background of project 
directors were gathered; (a) number of years of administrative experience, 
(b) number oi years of vocational education experience, and (c) number 
of years employed by the grantee organization. In general, the project 
directors were employed by the grantee organization prior to the start of 
the Part D effort. Of 44 directors for whom information wa*' a^^ilable, 
22 had been employed in the field^of vocational education for iiv^-ye'ars or 
more and ten indicated over ten years of vocational education experience. 
Thirty of the project directors reported more than five years of, adminis- 
trative experience. * 

Most projects did not elect to use the selection- of a * ' ' * 
director p^l^ion as a method of building a bridge between school and the, 
world "of w'ork. In addition, it is clear that projects we're not aware of^^or 
did not choose to follow the suggestion contained in the legislation that 
exchange of 'personnel between the schools and manpower a^encies^and , 
businesses be a part of the project design. ,^ 

I 

In terms of turnover of project directors, it was found that in 26 cases * 
there was a single director, in 15 cases there were two directors, and in 
three cases there were three project directors during the^ course of the 
project. In six cases* information was not reported. A comparison of the 
data pertaining to project directors with that regarding project activities 
does not reveal meaningful relationships. 

Dissemination 



The primary focus of this study was to evaluate the^degree ta^Which 
student outcomes attributable to project activities conformed to the legis- 
lative intent. In assessing the student outcome^produced'^y Part D 
projects it is reasonable to assume that if projects influencj^d other school 
systems, those school sys'tems could also be producing desirable student 
outcomes. The USOE apparently took this "multiplier effecl" into consid- 
eration when it indicated in the Federal Register that one criterion against 
which applications for Part D grants were to be judged was the inclusion 
of a dissemination plan. » ^ 

The design for this study included provision for collecting information 
with which to assess the influence of the pro-jects studied upon other 
schdols or school systems. Data were sought from two sources, -.the 
grantee itself and the sfate education agency. Grantees were asked if 
they had a plan for disseminating the results of the project. Twelve 
projects indicated that they did not have a dissemination plan and 38 
^indicated they did have such a plan. 
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Next," projects were asked if. their dissemination efforts w.ere formal, 
that is to say, written. Of the 38 indicating a dissemination plan, 18 
indicated that is was written and 20 indicated that it was hot. This means 
that 24%. of the projects did not have a dissemination plan and in 40% of 
the projects the plan was informal. It Wfts not possible to validate the 
grantees' evaluation of these'informal <Jis semination plans . 

Typically, grantees reported that they responded to requests for techni- 4? 
cal assistance, information^ and materials, and advice* Seve^^I reported 
the publication of a project newsletter as a part of the dissemination plan 
and several reported a^ttending conferences, etc. , as a part of the plan. 

Eighteen projects (3^o) indicated they had es'tablished written dissemina- 
tion plans. These plans included the development and distributiqn of 
career education materials, including movies, slides, and other audio- 
visual aids. Teacher training wpi^kshops and seminars were also a tech- 
nique included in s^eral of the formal plans, 

In general, the projects having formal dis seminaticn 'plans were active in 
disseminating results, materials, and other lessons learned as a result 
of project experience. Those projects having only informal plans, on the 
other hand, tended to be passive. In other words, they tended to re&pond 
to requests for'assistance or materials rather than generate interest^ 

There were few records ma^tained describing the actual level of effort 
.represented by dissemination activities. This made assessment of Impact 
impossible within the time frame of th^is study. No meaningful relation- 
ships between the presence of a dissemination plan and project activities 
were found. ' . • 

Continuation and Replfeation 

The USOE policy paper governing the first round projects required 
, grantees to make provision fpr carrying on the exemplary program with 
support from "tegular" funding sources after Part D support had ter- 
minated. In 14 cases, grantees indicated that proj'ect activities had been 
or would be discontinued when Part D support ended. Another 19 grantees 
indicated that some aspects of the project would be continued but at a 
reduced level. In most cases the reduction was occurring due to a iack 
of sufficient funds from "regular" -sources to maintain the Part D sup- 
ported level of effort. * 

Of the remaining 17 proj-ects, nine indicated that pivoject activities were 
being continued at a level greater than that achievedyith Part D support. 
Income cases this meant that the number of p^articipaVits was increasing 
but not th'e level of program support. In other cases/ the increase was 
attributable to the allocation of substantial local resources to continue or 
expand the exemplary program activities. The eight remaining grantees 
indicatec|^that project activities would continue but did not foresee 
substantial change In the nature of the activities or the level Of effort. ' , 



Grantees were also asked If they ^^Mpdered part or all of the Part D 
project suitable for replication. It^^s fourtd that only three p.rojects 
responded negatively, The 47 Indicating that all or part of the project 
was suitable for replication were asked to Indicate actual replicators of 
aspects of the program. Provision was made far up to six responses. 
Seventeen projects indicated six or more replicators of some aspect^of 
project activity. Six indicated no replicators, and eight indicated that If 
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ther^»verc any replicators they were unknown fc^fche project. No mean- 
► ingful relationshjlps between project continuation an^ replication data and 
project activiti-es were found. ; 

'3. Organizational Charajfteristics ' 

In addition to the nat§re of the stated objectives of the projects and the-man- 
agement practices tljey followe<J^, variations ^n the basic natuTe of the organi- 
zations receiving Pdfe{: Q g^rap ts^^ay help to explain variations in proje'ct 
activities. The Parf^Jf^gis'lation permitted variety in the types of organiza- 
tions or agencie^s permitted to. ope/ate Part D projects. The USOE was 
authorized to contract with LEAs, SEAs, and public or private, profit or non- 
profit agencies, organizations, or institutions for the establishment of Part D 
projects. , • 

Given the permitted variations with regard to the type of Part D grantee and 
the explicit intent of build ng a bridge between school and the world of work, 
it woul|4'be reasonable to expect grants to be awarded to a Vi^iety of organiza- 
tional types. \ In addition, it might be hypothesized that considerable emphasis 
.would be given to grantees outside the traditional educational systjoapn; in other 
- words, that construction of the bridge would not be limited to only one side of 
the chasm. 

It was found, however, that Part D grants were awarded almost exclusively to 
educational agencies. Of the 50 projects visited, 31^ ^2%) were single school' 
districts, four (8%) were educational service agencies sponsored by "several 
schpol districts, ten (20%) were state education agencies, and' tlxfee (6%) were 
institutions of higher learning. In total, 48 (96%) o^ the grants were made to 
educational agencies and two grants (4%) were mad^ to other^types^ of institu- * 
tions. One of the two grants not made to an educational agency was awai-ded ' 
to a public foundation whi'ch subcontracted the major portion of the project 
activity to a single school district, and the other'was awarded to a non-profit 
corporation which operated an experimental, alterna^^e.-sc hool sy'st^m in an 
urban area. , .-^ ^ 

in one sense, then, therk was relatively little variation among the first rou^d 
. '/-//■/v^^^'g.XAntefes ^^ From an orgp-nization and management perspective, however^ the 
htjtp6']?tant <Jistincti6ns bdtwe^en grantees with and without a dire^ct line respon- 
sibility for,, s'Ghool administration was also explored. In 32 prqjects (64%), the 
'v.*^ gran tee was "djtfjs.ctly responsible for project operations at the sc'hool level; 
^ \ ""^"^yi^a^/ii&^^t in^ ^e '31 projects administe red by local school districts phis the • 

'^jier3FiiLt/'$rer, school system case, tke Part D gfantee had a direct line, super- 
,^isory rel^'^i-oiisiii p^.^ith the participating schools. In the r-em^aining 18 

pir'Q'jects (36%), "the grantee had an indirect relationship with the f)articipating 
'* s.c'hdql'S through the local di.stricts. At this level of aggregation, hypotheses 
sugg^fe^s^iDg that grantees which have dire,ct, operating responsibility for the 
admikistt^atiop of schools will be more likely to produce the, Expected project 
activiti6';^-than other grantees are not supported. A-<ompar ison of the two 
groups. Trf, 'terms of project activities ( as reported in Tabl.e 9, Chapter VI) / 
does not^vjui^icate meaningful differences in performance. ^ 

Exam Lnatio»n,N>^ the relationship* of management practices to organizational type 
produced m^M.ed results. Of ten projects which were asse«sed as having 
implemented ;s,ound plans of operation, as well as having formal links to man- 
power agenc ie'S'^.nd third party evaluations, nine grantees were local school 
districts (the tenth was an SEA);of 40 projects asses'sed as having relatively . 
weak management practices, in 22, or 55.%, the grantee was a local school 
district. In general, while cons istent participant information' and expen- 
< diture data frequently was not readily available from grantee^s with an indirect 
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relationship to schools, also frequently was not readily available from 
local school districts, especially from their project staff, 

"5 

A review of the findings regarding expenditures, activities, and management 
indicators m terms of subcategories of grantees (e.g., LEAs, SEAs', service 
agencies, etc.), gener?illy did not produce meaningful pattern's . The clearest 
relationship identified bet\^'een organizational type and,aAy of the variables 
previously,discussed pertains to s tated pr ojec t objectives and the absence * 
of activities designed to rea^h these objectives. Earlier in this chapter it 
was noted that many projects stated objectives and subsequently did not i, 
initiate activ ities' des igned to reach 'those objectives. This phenomenon was 
noted in nine of the ten instances where a'n SEA was the<grantee. This sug- ^ 
gests that SEAs tended to overestimate the operational capabilities of educa- 
tion service^ delivery agencies. In other words, SEAs appeared to plan mjre 
than was possible with the limited resources. It also is possible that Yhis 
may be explained in terms of the management structure, the implicatjion here 
being that management adjustments are more difficult to make when 
grantee is jiot directly responsible far program operations. <i - 

C. Conclus ions 

The findings reported in this chapter addressed three basic questions whose 
answers might explain the variations among projects with regard to expenditures 
and activities reported earlier. The first of these questions was: Did USOE fund 
projects which were expected to produce the Part p activities called for by the 
legislation and policy paper? A review of Stated objectives in project proposals 
and reports indicates that while most projects did have the activities specified in 
the policy paper among their objectives, there wene exceptions. Although many 
projects included objectives which were in neither' the legislation nor the policy 
paper, none of the projects included#among their objectives those activities 
addressed in the legislation but not addressed by the policy f)aper. 



It should be noted that there were several instances of projects performing 
acj:ivities which were not included among their project objectives. Often this may 
be explained by the presence of objectives which wer6 too general to categorize. 
In addition, phased on discussion with project staff and confirmed by USQE program 
staff, in'some cases this was the result of USOE efforts to persuade projects to ^ 
undertake these activities. As indicated in table H, page 52, however, in no case 
does the number of projects with a mandated activity included ambng their stated 
objectives plus those who actually produced the activity equal all of the 50 projects. 
Also, as indicated in Table 11, with the Exception of *'elementa'ry and secondary - 
familiarization" activities," a cons ide-rible number of projects did n^perform 
activities included among their objectives. 

The' second basic question addressed was: Did projects comply with management" 
practices addressed by federal policy, and what is the relationship between these 
practices and engaging in the federally expected Part D activities ? Data "were ^ 
reported with respect to six areas of mana^^ement practice specifically addressed 
in federal policy statements. « . 

Typically, projects indicated compliance with some but not ^1 of the management 
practices reviewed. Of the 50 projects, ten were judged to have a sound plan of 
operation (based on positive respdnsq^ to three of fou«. indicators ), established 
formal hnks to manpower agencies, and implemented third par?y evaluations for 
at least two program years. Of these ten projects, five implemented all of the 
activities^pecified in the USOE policy paper which were included in..their set of 
local project objectives; of these five projects, three included all of the federal 
policy paper objectives among their own. 
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Analysis of the data cuUected with respect to project plans and efforts to dissemin- 
ate information to other school districts was not meaningfully related to the type of 
project activities in which they engaged. This wasr also true with respect to data 
regardin^project cuntinuation or to indicatiojis of the replication orf project activi- 
ties in oth^r school districts. 

o 

* Z' 

The third basic question addressed was: Does the nature of the Part D grantee 
relate to either the management prac tices followed or the nature of the activities % 
produced? In general, a comparison of the various types of grantees with the 
as se s sment of p roj'ect activities and management practice s did not produce jnean- 
ingful results. ^ ^ 

In suipnmarY, the findings in this chapter indicate considerable variability in both 
project objectives and^management practices , and that these variations do not 
closely correspond to the variations among projects with respect to expected 
activities. In Chapter V it was concluded that the federal inputs did not occur as 
planned as indicated by projecb^ underexpenditures . From this finding it would be 
logical to predict that the planned federal activity objectives would not be reached. 
In Chapter VI this was indeed found-to be the case. 

> 

In general, Ij^-sed upon the findings presented thus far, one would not expect the 
planned student outcomes to have occurred, at least on a uniform basis. In 
Chapter VIII, which follows, the student level outcomes of the first round 
Part D programs are presented. , * 
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CHAPTER VIII:. STUDENT OUTCOMES 
^ ' , A.O Introduction ^ 

The objective of the Perrt D program was to produce student outcomes as a xesult 
of program activities ^hich were consistent with the legislative goal of* reducing 
youth unemployment. While the P^rt D legislation and associated federal policy 
specified a set of broad program activities (see Chapter VI), the most explicit 
statements with regard to student outcomes of t*he Part D projects were:- 

* . <c- 

^ * »' • 

• that elementary an,d secondary students werO to be familiar wi^h the broad 
range of occupations and the requisites for ca*reers in these occupations 
(legislation); ' ' *J ^ 

• ' r . ' , . 

• "that secondary school *studen,ts were, to be motivated and prepared for 

careers as teachers of vocational education (legislation); ^ - 

• that students not enrolled in vocational education programs were to re- 
ceive'job entry skills just prior to graduaiion^ (policy p^er); and 

• that all students would be placed in a job or in post-secondary occupational 
training at the c6mpletion of their schooling (policy paper). 

Beyond these four statements^, the student Ijevel objectives of the Part D projects 
^were not formally Refined during the first three years of^the program. Officially, 
within the broad purf>ose of the legislation and USOE p61icy statements, each project 
wa^s free to specify its own set of, outcomes. During the course of the»first roCfnd 
activities, however, a set of more explicit expectations 'regarding student outcomes 
emefged and gained acceptance acrtoss most of the locally operated^Part D projects. 
These expected student outcomes gene;*ally may be traced to j;he report of the. 
Advisory Council'on Vocational Education (196^) which led to establishing the Part 
Optogram and to related conceptual and innovative efforts in education. 

The fin^iings with regard to student outcomes discussed below are, for tjie n^ost 
part, based on the set of informally specified but generally shar.ed expectations 
which guided , the first round efforts. The set of research questions ahd^hypofheses 
to wriich the findings relate were forignulated in the fall 'of 1973. They, are the re-"^ 
suit of a process which involved the active participation; o( the study team, PaVt D 
p rog ram' s taff, and ^valuatiop atafij-from USOE. Eac?h paretic ip*ant in the process 
was involved in formulating thfe outcome statements and approved.the finaf^set. 

As indicated in Chapter III, the set of student outcomes selected for investigation 
was not intended to be exhaustive of the <*utcomes produced by the ^art D prcJjects. 
Rather,* they were considered reasonable indicators of the extent to. 
which fede ral program objectives were attained.' * 

It was expected from the outset that not>all' p rojec ts would succeed in- achieving , y 
each of the expected outcomes. USOE program staff were aware tfiat projects / 
varied'both inemphasis and in the techniques they employe^. "The findings repeated 
in previous chapters with respect to project expenditures and c^ctivities also cij^rly 
lead to the 'expec tation that projects would vary in term^ of the student outcomes 
produced. ' ' ^ . / ^ . * * 
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B. FinSings * * '! 

. f '.i; 

As indicated abcrve, at the start of the study a set of research questions Ava^^ . 
developed j'.untly^by Development Associates and USOE. It*was understadd^ tjiat 
the answers tu these questions would provide an assessbnent of the perf^rfrjlanqe of 
t^e Part D program daring its first^years in tertns of student outcomes ,a.s they re- 
late to the national objectives. As will be notedtin the context of the subsections 
below, the analysis of the data obtained during \^}e field effort resulted in the modi- 
fication of a,fe>y of these que^ions . The specifi^l findings below are organized itx 
terms of The study questions Ipnd are based on the^responses of randckmly .selected 
students and, in one case, teachers,, following thel procedures described In 
Chapter IV. ' • ^ . 

For each study question, the findings are presented both in terms of the number af 
projectis in which there were ^^^^^^^re not positive fincUngs when comparing pa^rtici- 
pants and non-participants, and in terms of the differences between participants,*^ 
and non-pa rt.icipants across all projects. (It will be noted that at the 12th gcade 
level, the Work Experience Group and the Job Entry Skill Training Group showed 
fewer positive outcomes than did the Participating Teacher Group; an94.tjiat therd. , 
are several instances in which the Skill Training Group had less positive scores 
than its control group. , We believe that these findings reflet selection factors,\ 
since the questionnaire 'data indicate thajt only 26% of the SUill Training Participants 
and 29% of the W^rk Experience Participants, said thal;^ they planned to go to colleg,e^ 
in comparison w^h 48% of the Participating Teacher Pct^rt/cipahts , 49% of the 
control group used fai^Rie Skill Tl*aining comj^^ftcnt; and ^6% of tHfe control group 
used for the ^Participating Teacher component. Ti^e special control group used for 
the Work Experience component students in othej; c<iopL^rative education pro- 
grams -- contained 38% who 6aid they planned to go to G^mlfeg^ These data strongly 
suggest ♦that the participating students in the Work Exper^nce graupiin;d^ in the SkiU,^^ 
Training gr oup differed from the other students in their prior academic orientalion-^-) 

* ' * 

Question 1: Are student participants able to identify a greater number of ocjcupa- 
tipns than non-participants'^ 

C rite r la 

While it was judged that participants in the Part D program should be able to iden-. 
tify a greater number of occupations than non-participants, upon reflection it was 
decided thai it, would also be important to determine whether participants were also 
aware of a gre^-ter variety of occupations. As a rejnalt, two criterion variables 
were used to answer the above que stion. 

. la. Absolute number of cup at ions : 
f 

Each student was asked to "wr*it"etl^e^ames.ofja<^^any jobs or Occupations as you can 
- think of " on a^pags consisting of two cdiCtm^is of 24lE?iacnklines^ eacKcolurnn. *F.ar 
this criterion the scoring was in terms ef the njimber of itelhrifijis ted whi 
assigned a cod^ numb^ from the Dictionary of (Occupational Titles (DOT). * A t-test 
was used to determine whether a statistically significant difference existfed 
between the means of the two groups, i.^e. , participants afrd non-pa r tic ipants^,.> 
for e^ch student g"roup, on both an ove'rall and a project-by-project basis. 
For each test the null hypothesis was:^ Ha: pp;= y that is, no difference 

* exists between the mean score of the participant and non-particip'ant group. ^ 
The probability of falsely rejecting this hypothesis was^ set at th§ 5% level 

* (i. e. , <<; = . 05).'- Values at thfe 1% level were also i\oted. V 



lb. Variety of Occupations:- * ' -yj-^' ' 

The responses to questions yielding an indication of the number of occupa- 
tions^ were also analyzed to provide an indication of the variety of occupations 
reported by each student. Coding was in terms of the number of different 
two-digit DOT codes conta-ined in ^.ach studentl-s response. For example, 
while a student who ifsted all nine positions on a baseball team would have 
received a s<:ore of ''9'' in terms of the absolute number of occupations listed, 
he would receive a score of "1" in terms of variety. As with the first / 
criterion, 'a t-test was used to dete.»#hiae' whether there was a significant ' / 
difference between the two groups in each project and across all projects, j 

Results ' ' • ' 1^ ^ 

Table 14, pag2 66, shows that 6tlV2Tad,e participants,' in 15 of/Phe 45.pro^ects surveyed 
(33'^o)^, named significantly more occupations than did non-partic ipantsv Similarly, 
participants in 13 projects (29%) named'a s ignificajntly greater variety of opcupa- 
tions«=than non-participants^ > • . <• * 

I ' 

Ninth -grade participants in 1 1 of 42 projects C26%) named significantly^'more occu:' . 
^ pations and a significantly greater variety of occupations than did nqn -participants.. 'J 
*■ 

Participating Teacher (12th grade) group^participants nn 9 of the il> projects {39%) 
^ nai:ned s ignificahtly more occupations while participants in 8? i 3, projec ts (if^%) * 
a-' significantly greate'r variety* of occupations than non-*parficipant^ . 

CoUnseitng (12th grade ) 'group partic ipa'hts \n two of the ^ur projects w^.re'abte to > 
name .s'ijgnificantly more occupations, &nd* a significantly,greater Variety of occupa- 
y tionsvtKai/were non-participants . .'/ /.\ 



I 



Work Experience (12th grade) group participan-ts in 3 of thVT^.-projects (16%) wei^^//. 



able 

occupations than non-parjtipipants . 



to name significantly mjxte occupations and a significantly g reate r v4^ety/qf/.' 
pations than non-partipipants . /.*•// 1*^ . - 



' Finajly, Skill T.I^inu:^^(-mh^^rade) g ro>fp^py ticlpantS^ in^^ of the 14 p^pj^cis ^Jl'i%) * 
named s ignif icaTif^'jbfexe oc cUpations /nd a signilicaiitjy greater variety ^-tJccupa- 
ti(5fts than. non-par t5:ci6 ants.. \ ^ / ^ / /* ^ V' " V '^^ 7'^ " 

Generally, in pr<ia<jcts^hereVarti6;panijs'i^^ of - / 

. octup^ions than non-pajtie^aii'ts^/tb^y al^o^ identified a /jfgnificantly greater 
• varjety"?)f occupatiojis , /^O^^t!iye^Vexc^pt/i5 were found where participants score(J 

signfficaEtitly highe-r^in ^« vari^Je^l^at po"t/the otKeri ' • . . 

As sKown in, X'able 15, 5J9;5TparjLsWs w>Te' projects for eacfh group. 
4Parti€ii5a^ts" exhiHl^>^i^^pH>c^ feotli vajtiables^than did noh- 

participan^ irttJ^f^thT gfa<e and both 12t'h grade 

groups whiclv^d ocdupation^ Tamili4rizjaUo9^<tivi^i^^ infused into the curriculum 
(Participa>j^'g**Teach€r an^X^ouns elbr)^*'' For. ;tjbfe Work Experience and Skill Train- 
ing grou^^s, the'^non-paiticip^n'ts scored si^tttcant^y higher than the participants 
oh both' variable s'. ' " \^ * • • . ' » 

\ • - ^ i • ^ • - • ^ 



Conclusion: * o 

C-mparing results across all projects reveals that partic ipants scored significantly 
better than non-participants for all gr jups except Work^Exper ience and Skill Train- 
ing oipi both criteria. Although the differences between jth^e groups were statistically 
Significant, the differences were small except in the instance qf the Counselor 
groupl^vith respect to the number of occupations named. On a p rojec t-by-p r oject 
basis it must be concluded that in a majority of the projects, participants were not 
able to identify a greater number or a greater variety of occupations than non- 
participants. ^ 

Question 2: Do btudents demonstrate more familiarity ^ith tasks and functions 
associated with selected occupations than the comparison group 

C rite rib 



me 



Each student was asked to complete Part 2: Knowing About Jobs of the Career 
Maturity Inventory. Each of the 20 items^n this sub-teSt begins with a brief de- 
scription of a job performed by a person. Following the description ^re folr 
occupational titles. Students are asked to select the occupation corresponding^to 
the job description or to indicate that the^ "don't know/' Scoring was in terms of 
number right as a percentage of" the number attempted; if fewer than half were 
attempted the student was treated as a non- respondent. A t;-tesjt was used to 
determine whether a significant difference existedibetween the means of the two 
gifoups, i. e. , partici^nts and non-participai^s, ;for each student grpup on a project- 
by-project basis. For each test the nuirhypothes is was: Ho: pp = pnp ; that is, 
no difference exists between the mean score of the participant and non-participant* 
group.- The probability of falsely rejecting this hypothesis was set at the 5% level 
( SC = » 05). Values at the \% level were also noted. 

Results 



As shown in Table 16, s^xth grade participants in nin^ of, the projects (20%) scored 
higher than non-participants . Ninth grade participants in 11 of the projects (26%) 
scored significantly higher than non-parTicipants . Participants in only one project 
in each of the 12th grade gr^oups scored significantly higher than n(5n-participants. 
Table 1^7 shows the *comparis ons of participants and non-participants t)ve r all projects 
Participants scored significantly higher than non-partic ipants -in the 9th grade, . ' 
Participating Teacher (12th grade), and Counseling (12th grade) groups.. No differ- 
ences were found in the 6th grade groups. Non-participants scored significantly"^- 
higher than pa r tic ipants in the Skill Training (12th grade) group, while no difference 
was detected in the Work Experience group. ' 

• 

Conclusion 

The area of greatest program impact with respect to this jutcome appears to have 
been at the 9th 'grade level where participating students scored significantly higher 
than non-participants in 26 %' of the 42'p rojec ts, and across* all p'rojects. "While 
on an across-pi^^ject basis participants in three of the six student gro^ups scored 
higher than non-parHcipants , in a majority of projects participants did not have ^ 
significantly higher scores. Overall, it is concluded that participants did not 
demonstrate more familiarity with selected occupations than «on-''particip*ants . 



Question 3: Are student par ticipants more familiar with the requisites associated 
with employment in selected occupations than the comparison groups 

Cri teria . • ' 

Two s^ib-scales of the Careej" Maturity Inventory (CMI) were usjed in answering " 
th*6 que^stion. While they are the best measures relative to this qviestion which 
cadld be found, it must be noted that both were designed to measure more than 
simple familiarity with the requisites associated with selected occupations. 

- frhe first sub-scale was spe^:ificaHy developed toass'ess a student's ability to 
'match up" psychosocial characteristics with those required in selected occupa- 
tions. The second was specifically developed to assess a studSit's competency in 
utilizing specific knowledge of career r^equisites in planning. 

3a. Each student was asked tlb^mplete Part 3; Choosing a Job of the CMI: 

^ In each of the 20 items in this sub-te'st students are given a description of a 

person, followed by four occupational titles. Students ar^ asked to read the 
description of each person and ther^select the occupation they think i« the 
"best'^^ne for that person or to indicate they "don't know." .Scoring for this 
and all Wher sub-seal^ of the CMI was as described for Question 2. A t-test 
was used\to determine whether a significant difference existed between the 
^ means of the two groups, i. e. /participants and^non-participants, for each 

student group on a project-by-praject basis and across all projects. For each 
test the. null hypothesis was: Ho: pp = Pnp; that is, no difference exists 
between the mean score of the participant and non-participant group. 1 
' * probability of falsely rejecting this hypothesis was set a t the 5% level {J^ = .05). 
Values at thh- 1% level were also noted: - ~ 

3b. Each* student was as^ed to complete Part 4: Looking Ahead of the CMI: 

In each of the 2^0 items in this sub-test students were given an occupational 
title and three steps tJiat a person could complete to prepare for an.d enter 
this occupation. StuHents were then given four ways in which these steps 
could be^or-dered. JEach student was aske^d to read the occupational title and 
^ ' the steps and then select the correct order for completing the three steps, or 
'to indicate "don't* .know. " ' - u 




As with the first criterion, a t-test\vas employed to determine if a significant 
difference-e"3?isted between participants and non-par tic ipattts on ° 
prp-ject-by^pfQjeojt basis and a^eross/all projects. The null hypothesis \yas: 
Ho: pp = Pnp for each project and afc»«^^ll projects. Again cC was set at^ 
>ithe .05 level. . ' . * • " . 

. ^ ' . » * ' ^> - 'J " ' % 

Results ~ . ^ • ' 

.^As indicated in^able i^", partic^jyants in seVen^proj ects (16%) at the sixth grade 
. Jevel h?td s^lfnificantly highe»r jjiedn dcc>res ori the Choosing a Job sub-test. On the 
suit)- test LooT^ing Ahead , .s^ixth^gjade participants in ITprojectf (24%) had signifi- 
cantly hi^^^T'jneap scares ^tha'n yilfyr. tonjparisbn groups. ' 

At the ninth grades, tpar titt^ant^ in 10 projectg (24%) on the Choosing' a Job sub- test 
and in six projects (14%) on the LooIcingfAhfead sub-test obtained, significantly hTgher 
mean scores". * % ' * 

• ■ - V-/, 
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At the l^S^^^jJPiicle level, participants in the Participating Teacher groups scored 
higher tfeap non-participants in three projects (1^%) on Choos ing a Job an<^ in four 
projects (17%) on Looking Ahead ; par^apants in the Counseling group scored * 
higher, than non-participants in one p.r^Dject (25%) on both sub-tests; participants 
in the Work Experie^nce group scored Jiigher than non-participants in four projects 
(21%) on Choos i ng a Job, and in. two projects (11%) on Looking Ahead ; and partici- 
pants in the Skill Training group scored'higher ixi one project (7%) on Choosing a 
Job and in no projects on Looking Ahead . As indicated on Table 19, when compari- 
sons were made across all projects, participants w^re found to have significantly- 
higher mean scores In only the 6th gra<3e group. 

ConcluS ion 

In comparing participants ta non-participants across all projects it is c(m€luded 
that only ^t tiie 6th gi'ade level did the participants score s*igrvificantly higher on 
the sub- tests than non-participants. However, this was found to be true- in only ^ 
seven of forty-five projects (16%) for one criterion and 11 of 45 for the second (24%). / 
In general, then, it?nust be^concluded tha'n pa rticipants were not ntore farjiiliar Jk 
\vith the requis ites assoc iated with employment in selected occupations than non- 
participants. / * ' ^ - 

Quest i on 4: Do student participants score higher on pre-vocational, J^ob readiness 
tests than non-participants ? , 

Criteria ***** / 

> i ^ ^ . 

No standardized instruments relative to this que'stion were" identified. As a result, 

two sets of questionnaire items were developed. During the pre-te*st site visits, ^ 

students at each grade level were asked to complete Ih'e items and discuss them 

with ^member of the stiidy team. The items were also^reviewed.for face* validity 

by staff of Part D projects during the field test visits and by individuals with 

extensive experience in the'-/ield of job training and placement. 

4a. The first set of items, designated Job •^eadinn^s; ^nowled^e , ,was designed 
to assess information pertaining to the world of work which indicated job 
readiness. Twenty factual statements to be checked as either "true" or 
'•'false" (Questions 1(5 through 29 of the Student Questionnaire) 
were asked of each 6tukent.^ The'test was scored by giving one point for each 
correct aj]|^wer and'summing the points. A t-test was used to determirfe I 
whejther a significai^t difference existed between the means of the two g-roups, 
i.e. participants and norf'-participants, for each group on a project-^y-p roject 
^ba'sis andacross all projects. For each test the null hypothesis was:, Ho pp ^"^Jnp; 
that i^, no difference exists between' the mean score of the participant and non- 
participant group. The probability of falsely rejecting this hyp'othesis was , 
set^at the 5^o level (i.e.j = .05). Values at the 1% level werei^, also noted. 

4b. The second set of items, designated Job Readiness: Attitudes , consisted of 
seven statements pert^ning to various preferences and atUtudes concerning 
work and career planning (Questions 30 through 36 of the Student 
Questionnair-e).* . The. test was scored by rlneins of a five-point agreement 
scale, with the scale reversed for items in which disagreement was the mor(^| 
favorable response. The score for estch of the items wae then summed for a 
total score. *As with the first criterion, a t-te^tTwas employed to determine 
the difference between the participating and n on -participating groups on a 
project-by-project basis andacross all projects. ^ 
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Results 

- % 

J ob Readiness :> Knowledg e - ^ • 

The results for this^ aCnd the second criterion are shown in Tables 20 and 2 1 . At the 
sirfth grade level, participants in 8 of 45 projects (18%) scored significantly higher 
than non-participants. Participants^n the 9th. grade in 7 of 42 projects (1,7%) 
scored significantly higher than non-participants. At the 12th grade level, parti- 
cipants in only one project in each of the Participating Teache r and Counsejling 
groups scored significantly higher than non-participants I In the Wb?:k Experience 
group, participants in only one project scorned signif icant|ly highe^ than non- * 
participants, while i(o^ignificant differences were found in the Skill Training group. 
On an overall' basis , significant differences^^vJ^re founfj only in the 6th grade group 
and these differences were, in a practical sense, very sraalli No differences 
Were found in any of the other groups. 

Job Readiness:- Attitude 



Als shown in Table 20, *parti£^aiits in four project^ in each of tM^ 6th and 9th 
)^r;:ade groups had significantly nigher nieans than non-participants. At the 12th 
grade level, participants in ttiree projects (13%) in the Participating Teacher group 
and on^ project (25%) in the Counseling group had s"l^nif icantly higher means than 
non-participants. In the Work Experience and Skill Training groups^ particiipants 
^scored significantly higher in ^ne project (5%) and jio prt>je^ts , respectively,. On 
an overall bajsis, significant dif/erences were foujid between]*^^rticipants ajid^non- 
participants 'at the 6th grade level(and in th^ Skill Training "group; in a prS&ctical 
sense, however, these_ differences w^re very ^mall. ^ No differences were found in 
the other groups. ^ c ' 



Conclusion 



' On both an across-all-projects basis and on ^ wi^in^projects basis, some statis- 
tically significant remits werfe found. Th^ greatest area of impact appears to haye 
been at the sixth grade level where a significant difference between participants and 
'non-participants was foupd overall and on one or the other of the criteria in 22% 
(lO)'of the projects. Given the small number of project in wlii^h differences were 
found plus the lack of significant differences overall in grades 9 and 12, it must be^ 
concluded that»" in general, .the res-ults do not indicate that participants are able to 
gcore higher than non-participants on pre-vocational, job readiness tests.^ This 
coh«Jusion must be tempered, ho^^^eyer, with the observation that the scores of all 
student^ at all levels, were C[3^ite high in most projects* * " 
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Question 5: 



Do student pafticip>tit>s:indiyVee niore po&itive attitudes toward employ- 
>m^t ti)4n^non>'pa'rritipant^^^ : . w> 



■ . The Attitude "Scale of the Care^ti iCUtu-ritt Invexit'ory was used as the indicator of 

\ ; \' - Y \P?sitive attitudes toward employment. The Altitude Scale consists -of 50 state- ' 
' ; / ' ^qnis peT4:ainin^ to iQ(:iiyi,ci«^j;;''^ttitu'de.s a^ feelings. tov^ rd-making a career 
, . ?^,S:^^^orin^ Each s tudei^t'^s' ask^d to in<ijlcate whether 

.'^"^^;^|rf^^'^">^^g^''^^^^^ As with the other sub-scales of the 

..V V OMl, scoring wa.s ii^ 4iemis -^;ptefc^"n\a5o. right, if more than half of the items were 
att^mpJey^if^e^er thah -h.aU!^:^^^^^ €b^ student was treated as a non- 

' ^respondent. <\ t-res^Was.iT^%.to/letVp:irhe>h4her,a significant difference existed 
^ . betwe.^n thei^eans^of^t^.^v'^.t^i^o^^^^^ i. 9- Vpatticipants and non-participants, for 

\ ^ ; . ^."'-'^^h^.^i'',^^^'^^^^^^^^^ aTid across all projects. For each 

X/r:.- i^^rf^^'t^^'n\,i\^hfp^ Ho: ^.p -f^W;'t>>it is,, no diffeience exists between 

the mean score of the two groups. The t^Vob'^bUit^' ofrfalsely rejecting this hypothesis 
was set at the 5% Iwel \ cC .'aides- at t^'^.l^o level were also noted. 
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^Ta^le22'-.hows thet .it tfie s;ixtn '^M^le ^eve> pa.f ticiVants In -3 1 projects f24%)r had 
^^significantly higher ^j^ean^Co Lies ^ rAtJ;be ninth grade level, 

• , pahicapant^ sqored> ifeniitcji/tly.hi^iefr '^^^ (14%). 
^ At fMl2th'gra,de le-v^l,' ^rtUrip^tin^ ^eacher-a^nd Goiiiise^ri'g •griup^participants in 
. // ' th-ree^r.njeciswfl-srdl ind'^iie pr,bjVc< (25*^^ r^spec^Iv^y, likd signljt^ higher 
/ (11%) in the Work E:^er- 

' ' '^y^^'^^ny^^Y*^,^^^'^^^^ non-particiiiants; no 

' ! ^^f^? <3?tect^^,inJh6.S?:2ll'Trai^ Table 23 shows the results 

. A/t^f? ^9mj^a/i^c^^Vaiv^;t^^ pVo]ec,rs. ^Significant differences were 

^ ;^ - -found at the/sd^i.t^ and jiv'tKQ.Pa-rticipating^^^^ group. No differences 

• were fpu^d'm^the other'gr bCips '. V-'-i-'^' ' ' 

^. --7^^^- /^/V^^ across projects,, it is foun^ that sixth 

''"^^> ^/SVa<j<>'am.^j;^''!j^ participants indicate'd^a.mpre positive attitude 

^ ^^^^Jl On a pVoject-by.^r^c^jfect basis, however, 
/ 'Uarticip3?f£s 'vhj.^x^^^^J^ of ejects did not indict tp a,^mo?&.^os\'tive attitude 

' / * ( ^oy^^t^hT(\^\TSy^^ . Taking the -withln^ancF ac ros s-proj ect 
'^•/^ '^. V- '^'9^^^P*^^?'°"s (oJJ^'^eVrJ^i^-coric^ that in general, participa^nts did not have 

' ' \ -V, y.'- {D'ore p/>^itiv<*;Attitii?Je?^oyard Employment th§Ln noti'-participaiit^^^ 
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Question 6: 



C riterion 



Is the variety of careers being conside-r-e4^ by individual participating 
students greater than that of non-participants' ^ 



Each student was asked.to respond to the question: "What are all the different 
kinds of.work you might like-to do when you grow up? " This was question #9 of 
the stu<;lent questionnaire. Responses were coded to reflect the 
oFrtimber of different two-digit DOT' job codes included in each student's response. 
The categorieTused for analysis were 1,2, 3, 4, and 5 <3r more different two- 
digit DOT codes. Results were analyzed using the chi-square test. 

Results « 



Participants indicated k greater variety of careers than non-participants in four ^ 
projects at the sixth grade level, oneprcject at the ninth grade level, and one project ^ 
each for the Fkfticipating Teacher group, and Woxk Experience group. As shown in 
Table 24^ when comparisons were mad6 across all projects, no, difference between 
participants and nojri-participants were found except in the Counseling group where^ 
participating students cited a smaller variety of career choices than the •non- 
participants . This suggests that one of the effects of counseling for thi§ group 
may be to narrow the career choices of the participants. Overall, less than 50% 
of each group cited more than two different DOT codes. 

Conclusion 

It must be concluded that participants are not considering a greater vai*iety of 
careers than are non-participants. 

Question 7: Do studenjb participants indicate more positive attitudes toward 
guidance and counseling than 'non-participants 

Criteria . 

1 I- , ' , - ^ • ' 

All 9th and 12th grade students werd asked whether they would like to meet with 
their counselor more often (Student Question If 48). In addition, those 
who indicated thslt a counselor had t|tlked tovthem individually about their future 
work were asked if this had been helpful to' -them (Student Question #43a), and 
those who indicated that a counse^p^r ha'd talked to them individually about how to 
prepare for their future were as^ed if this had been helpful to them (Student Ques- 
tion ^44a). Twelfth grade students were asked ^iree additional specific questions 
in the same vein, dealing with helpfulness of a Jtinselor having talked to them 
about getting a job, applying to callege, or getting vocational or technical training 
*(Student Question #50^. All of the items call'for'No /Yes responses. Comparisons 
were made between participants and non-partiqipants by means of chi-square, using 
the 5% level of confidence for rejecting the null |iypothe sis of no diffexeirce'. 
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Results 

The results for all six variables are shown in Table 25, No significant differences 
were founciacross all projects between participants and non-participants on any of 
the variables. Within the 12th grade groups, b'etween 49% and 61% of the partici- 
pants indicated that they want more frequent, cdntact with counselors; between 71% 
and 83% of the participants arid non-participants said that it had been helpful to 
talk to their counselors abo^ut th^.futurej between 84% and 94% said it had been 
helpful to .talk to their couns(^16"rs.^about preparing for future work; between 74% 
and 87% said it hsfd been helpful to talk- to their counseloa>S about getting a job; 



* Dictionary of Occupational Titles. 
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between 69% and 89% said it had been helpful to talk to their counselors about 
applying to college; and between 72% and 83% said it had been helpful to talk to 
counselors about vocational or technical training. * , , * 

On a^project-by-project basts, very few significant differences were found between 
- participants and non-participants. 

i 

Conclusion * >■ 

Both participants and non-participants held very favorabl^ttitudes toward guidance 
and counseling and indicated they would like more frequent contact with counselors. 
No differences in attitudes were found between participants and non-participants. 

Question 8: Do more student participants indicate having a> career plan than 
non-participants ? , « 

Criteria I ' 

At all three grade lieyels, students were asked to choose one of nine alternatives 
describing the plai& they have after high school {Question ^4 in the Student 
Qoestionnaire); the^^responses to this question were then analyzed in terms of the 
^"don't know" option. ys. all other options to identify those who did and did not have 
a career plan. In a*ddition, students at the 9th and IZtfi grade were asked directly 
whether they have a plan for preparing for the work they want to^o in the future | 
(see Question #49 in Student Questionnaire); this was a Nor/Yes response option. ^ 
Resutts were analyjied using the chi- square test. 

Results I * " " 

Within each student^group across all projects, small and very similar percentages 
of participants and .non-participants chose the "don't Icnow" option. The pe^'centages o 
were 15% and I2^o rjbspectively, at the 6th grade; li% and 10% at the 9th grade; and 
between 4% and 9%Jbr the various 12th grade student groups. 

4 ■ • . . . 

The results for thc^direct question "Do you have a plan. . . , " shown in Table 26, 

indicated a signifit:ja,nt difference between participants and non-participants only for 

the Participating ^iic her group. In that group, 66% of the participants responded ^ 

affirmatively, whilf^^only 60% of the non-participants responded that way. No C 

differences betwejiii the participants and non-participants were found- in any of the 

othe regroups. In^ner^ between 56% and 66% of all the 9th and 12th grade 

students indicatedSliat they had a career plan. 

*As expected, give^:he small number" of students in each group who chose the "don't 
know^' option, ana^^is of the first criterion on a within-project basis did not 
produce meaningf^results . On the other indicator, significantly more participants 
said they had a pl^Hithan non-participants in o-nly two projects for the 9th grade 
groups and in onl>^^e project in the Skill Training (12th gi:ade) group. No other 
differences were mund. 



Conclusion 



The findings indices that well over half the students indicated that they had a plan. 
While a significant ifference between participants and non-participants in the 12th 
grade Fkrticipating^|reacher group was found on an across^project basis, analysis 
of results on an Jmfvidual, project basis produced no significant differences. In 
general, then, it ij^concluded that there ^diS little difference between the propor.- 
tions of participar^^ and non-participants who indicated having a career plan. 
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QuesUon 9. Do more student participants include further training or education 
as a portion of their career plan than non-participants '> 

Criterion " t 

Students in the 9th and 12th grades who indicated that they had a career plan were 
asked to describe their'plan on blank lines in the student questionnail-e (see Student » 
Question ^49a). Each response was coded by a DA staff rhember to reflect whether 
it included further training or education. , ^ ' 

Results 



A very s»iall percentage of the plans of both the participating and the np^Eparticipating 
students failed to include further, training or education. The percentage^^pif stu- 

« dents indicating that their plan did include further training ranged from approximately 
95% of both the participants and non-participants in the 12th grade Participating 

^ Teacher group to approximately 88% of the participants and non-participants in the 
12th grade Work Experience gl'oup. Approximately 92^ of both groups (i.e», par- 
ticipants and non-participants) of ninth graders indicated that they planned to ^ 
receive some training after high school. 

There was no meaningful difference between participants and non-participants for 
the 9th and 12th grade student groups within any project* In only one group in one 
pfoject did the combined total of participants and non-participants who said^they 
• did not plan further training or education equal as many as 20 students and in that 
case the rqsponses were approximately evenly divided between the two groups 
» ' * 

a 

•f 

Conclus ion » 

There is no difference between participants and non-participants with regard to 
this question. In general, the great majority of students, i-rrespective of project 
participation, indicated that their plan for preparing for work they "want to do in 
the future" do&s Include further training or education. 

Question 10": Do more student participants cite their career preference as their 
expected career than non-participants '^ 

Criteria 

In addition to being aske'd to cite "the different kinds of work you might like to do. . 
(Student Question #9J, 12th grade students were also afeked what'kind of work they 
think they will be doing "10 years from now" and "20 years from now" (Student * 
Question ^41e,d). The response to the career preference question and to the 
expected work questions were compared for each student b^ DA coders, who noted 
whether any DOT two-digit code in the former question was. the same as any- DOT 
two-digit code in the latter questions, difference betwe^en the participants 

and the non-participant groups in the extent to which career preference was the 
same as career expectation was then tested by means of the chi-square technique* 

Results 



As shown in Table 27, on an across-project basis no sJ^gnificSint differences were 
found between participants and non-participants in the percentage of cases where 
students' choice of career preference' equals career expectation^in the* Participating 
^ Teacher or the Work Experience groups. Significant differences were found, how- 
ever, in the Counseling and Skill Training groups. In both these groups, the partici- 
^— pants had a greater percent of agreement between career preferen^^e and career 
expectation. ■ ' * • ^ . 

* 

T 87 - 



In addition to i-oniparisons bet\\een the responses of participating and non-partic;ipating 
students on this criterion, the extent of the relationship between career choice an4 
expectation wl^ihin each of the participant and non-partifipant groups is oi interest. 
Similar career choices and career expectations were stated by 62% of tjt^e partici- 
pants in the Counseling ^roup. Agreement varied between 45% and55<fo in each of 
the other IZth^grade groups for both participants aifd noa- participants. 

- r « ' 

When the results were examined on'^a within-project basis^ significant differences 
between participants and non-participants In the extent of agreement between 
Career preference and career choice were found in only fhree out of 14 projects (21%) 
in the Skill Training group and 't<vo of 23 projects (9%) in fhe Participating Teache r 
groups, ^ , ^ , * - ' 

Conclusion ' * ^ 



■ ) ■ 

In general the findings suggest that there ijs little relation bet veen program partici- 
pation and the inclusion cff students' expected careers among -those they "might like 
t'D do. " The possible exceptions to this conclusion when vi^ed on an acros s-pr oject 
basis are participants enrolled in Job Entry Skill Training and Counseling grau^«-.^ 
In general, a large proportioij of the seniors surveyed, regardless^f participant 
status or sampling group,' did not include theii^expected ,caree r among the list-of 
what tliey "might like to do.'' • , ' * 



TVBLE 27 

Com>»aruon Ba^-vtcD Pvtrucijiit is (T) inil Njn-*'»n;i:ii>^na (.\P) o\ Student Cwup. AcrOi^ -11 Projects, for <?. 10: 

'*>o Tnotf tvad^ nt pTrticvp-nts cite tJicir career profc-cin-c as thfir expected cjrcei' th .rr no i-p^rtic pints ^' • * 
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Question 1 1 - Do more stadent participants cite vocational education as t^eir future 
ca/eer than the comparispn group^ '^ * ~ 



Discussion 



The responses of each 12th grade student to the question of the kind of work they 
think they w^U be doing in the future (Student Question Mc,d) were coded t^ indicate 
whether a job ^ssocig^ted with the field of vocational education was cited, tt was 
found that, on a national basis, only 12 p^articipants.and 5 non-participants cited 
some form of vocational educa^on as their future career. These frequencies were 
too small to warrant further discussion, v » . • ^ 

Question 12: Do students evidence behavioral gains as,a result of participating in 
part D project activities ? > . 

» » 
Criteria o 



Initially, it was hope(i that direct comparisohs,betWeen participating ai«d non- 
participating students would be possible^, Howe\{er, procedur;6s for these 



comparisons could not be developed, and as a r^^jil^^ was decided to obtain an 
indication of the'effect of project activities through a series of questions asked of 
participating teachers. Specifically, teachers were asked {6t their opinion of the 
effect of "the inclusion of career education activities" with respect to tardiness, 
absenteeisrrf, discipline problems, and general interest in school. For example, 
each teacher wa§ as^ed , whether career eduQation ac1;ivities increased and decreased 
tardiness* \ 



. The responses were analyzed in terms of the propcartion of teachers in each project 
who said there was an increase and decreas'e in each of the four behaviors. In 
addition, the responses of all teachers over all projects \^ere analyzed to determine 
wJiether there was a difference between teachers who primarily taught students at 

*the elementary grades (1-6), middle school grades (7-9), and senior high school 
grades (10-12). To asses^ the differences between teachers at the three grade 
levels, the following hypoth.esis was investigated: 

Hoi: The proportion of "yes" responses by teachers in*the elementary, ^ 
middle, ai!d senior high school grade levels is the same. The chl- 
« square test was employed with = s05. , - ^ * 



Results 



Table 2.8 shows the ^percentages of t*eachers at the t^iree grade levels that indicated 
that there was a beh|^or change on the pa^t of students as a result of "the inclu- 
sion career education activities. " 



1^' 



TABLE 28 \ 
Percentage of Teachers who Indicated that Career Education Activities Resulted in Certain Changes in Student Behavior 


Behavioral Changes * ^ * 


Elementary 
School Teaohets 


Middle 
School Teachers 


* Senior High 
^^^chool Teachers 


Increased Tardiness 
De.creased Tar^ines^ , * 


' 0.8% 
12. 5*/^ \ 


2.2% 
21.0% 


^ 5.7% 
26.9% 


Increased Absences ^ 
Decreased Absences 


1.0% 
19.9% " ^ 


3.7% 
22.9% 


8.6%. 
31.6% 


Increased Discipline Problems ^ 
Decreafsed Discipline Problems 


3,09^ 
33. 394 


7.2% 




7.5% 
36. 6% 


Increased Intere^ in Schocil 
Decreased Interest in School 


70.1% * 
2.2% 


70,1% 
3.4% 


70.0%" 
, '4.6% 



a) Tardiness: The results' showed that '0. §%', 2.2%, and 7% of elementary,, 
middle, and senior high school teachers, respectively, said that career 
education activities resulted in an 'increase in tardiness. Oi\'the qther hand, 
12.5%, 21.0%, and -261. 9% of felemeiitaiyr middle, and senior high school 
teacher?, respectively, said that career education resulted in a decrease in 
tardiness. 'Differences at the^.'. 05 ijevel of significance were found among the 
three ^rade levels in both variable^. In 48 of the 49 projects* surveyed, at • - 
least one teacher said that the inclusion of career educatidn-'activities decreased 

*ta^dine^s; irf three projects, ove^ 50% t>i the teachers indicated that career 
education decreased tardiness. The number of teachefs who indicated that * 
career education increased tardiness was, in eacK project, always less than 
20%'Of the responding teachers. 

b) Absences: R^j^s .showed that 1 ..0%, 3..,7%, and 8. 6% of el ementarVi 
middle,»and senior high school teachers, respectively, said that career ediica- 
tibn actlvities^resulted in an- increase in student absences. On the othe'r hand, 
19.9%, 22.9%r/^nd 31. 6% of el^menta^ry, middle, and senior high school 
teaqhers, respec^:ively , .said that career education resulted in decreased 
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absences. Differences at the .05 level of significance were found among the 
three grade levels in both variables. In all 49 projects that w«re surveyed,* at 
least one teacher indicated that the inclusion of career education activities 
resulted in decreased absences; in five projects, over 50% pf the teachers 
said that career education decreased absences. ' The number of teachers who 
indicated that career education increased absences was, in each project,^ 
always Iqss than 20% of the responding tesichers, t * 

*' » ^ - 

c) Dis'cipline: Results showed that 3 . p%, 7.2%, ajid^7. 5% of elementary, 

middle, and senior high school teachers, respectively, said that career , 
"\ educ'ation activities resulted in increased , discipline problems. The dtfferences 

among the grade levels are significant at the ,05 level. On the other.hand, 

33.3%, 39.1%, and 36.6% of elementary, middle, »and senior high school . 

teachers, respectively, said that oareer education resulted in decreased . 

discipline problems. No significant differences were detected across grade 
leVels on this variable. 'In all 49 projects that were surveyed, at least one 
teacher indicated that the inclusion of career education 4ecreased discipline^ , * 
problems; in nine projects, more^than 50% of the teacher s^surveyed said 
that career education decrea'sed^discipline probliems*. The number of , 
teachers who indicated that career education increased discipline proBlems 
was, in all projects except one, always less tha,n 20% of the responding 
teachers. • 

^ ' ' \ 
d) Interest in School: Results showed that 70. 1%, 7l()/l-%, and 7a. 0% of' 
elementary, middle, and senior bigh school teachers, respectively, said • 
that career education ac^vities increased ^studefiit inter esV in. s'chool. On 
' the 'o.ther hand,'2.2%, 3. 4%, and 4. 6% of elernentary, middle, and senior 
high school Jteachers,' respectively, said tH4't career educatioh decreased 
interest in school. No significant differencres artxong the grade levels were 
found on either variable. In all but one of theX9 prpject^s^^that were surveyed, ^ 
50% or mor^ of the teachers indicated that ipterest in 5'chool increased. In 
owly 53% of the projects did any teachers indicate that career educatioi\ 
activities decreased student interest and in^'SO of these 26 projects this view 
was reported by only one teacher.^ // ' * \' 

Conclusions ' . ^ .'^ ' V . 

Overall, the four sets of responses suggest ^at^tht^flnclusion of career education/ 
activitiea, in the opinion of teachers, k^A^'^o^^.r&XriQ.x^^s^^ interest in school i^^, 
to a lesser extent, decreased discipline p'r^bl^yns, ^ibsences, and tarcjiness"./^^ ^; 

, <*' V" , • ' ' ' 7 / 

Question 13: Are a greater nuhnber of stadeAts wlio have graduated from s^hj^ol a«i>d 
who participated in the Ps^r^ profgct employed full-time or ^i^^^aged 
in further training than st.ifjen^s who ^id'not .participate ? ' ^ / \ ^ / 

Criteria ^ > v " ; / / / / /; . ^ . ' / ' , . 

Time and resource constraints did nt>tyipe'r.nsij^ s/ study design providing fcJr ^he 
actual follow-up of graduates.^ RatheH .t^^'j^esign relied on' informatipn which ' 
was to be obtained' from school recoirds-i.'. ^^ecifically, ii>fqriihation.^w^s sought 
with regard to the work" status during t^(i*/i^r 4 year after gVaifiuation oi students 
who graduated from $chool the year pr.io^ to the initiation^^ tlie Part D project 
and of students who graduated at the eitd,' of/the 1972-73 ^chocil year. Comp^ris^bn^" 
were to be made of changes which occtcV^6^d,over time for participating a^ nop- 
participating students. ' / -'^ ' ' 




Th^e lacl^c^ Informatics wTtK te^^a^rd Jo^tju&'^ueatioii is cbnsit^'r^ed^a^n/itn.pgtftant 4/ 
find i ng~a s It 'j^EJjvqt^ be " po s ^-ilife j 0^0 s se^ th e e f £e c f , 6 £ Pa^t Qe "r f or^t s 

on graduatjs^^ In^jtji^i futi^re jilithput .t'eii^Me. placement r'ecjord^ 0;r p^ovi^ion for . , 
' extensive student Tfot^lo^VT-jj^'^pO^^^^ .r, \ r . ^ ^ 

^i'%^.^''^ff ' 'v</« , bu^m pi>ary '/an^ "GonSlusiong^ : - ; *v ^ 

The fttidtngs with^^especl: tp the, oatc^iine/^^Ufe^M^ which were addressed in this 
^ chapter present a mixed pictur^e of pr^ir'atn^i^^ When comparisons W^ere made . 
across all projects, .participants w<>r^e fbund to soore ^significantly .hrgher'ti^^n poii^ 
participants on some outcome qu^.st^i'^jhs while scorirtg tlt^ Same*nr IpVver pn^ others. 
On a vvithin-project basis, differences were found favorable to-Spme 'proj€ots,-bu^ 
not the majority, '•on most outcome questions; in pi any c^sesY either no differences 
were found or non-participants were found to sco.re better, . These mixed findings 
are further complicated by results which were not co^si^^tent over- the three grade 
levels nor over the* four 12th grade groups. . ' • ' ' 

Table 29' summarizffs the results with respect to fhe outconie, questions, acfo^s 
all projects, by student group. , Five of the thirf^en outcome questions have been 
omitted fron> the summary. Question 13 was omitte<? because insufficient data wei'e 
available for analysis. 'Question IKwa^s omitted because ther^ vvas insufficient 
vajri^bility in the responses of participants and non-participants to permit analysis; 

^ i.jp., over 95%^ of both groups responded in the negative. ' Questions 7 and 9 were 
^initted from the sumrpary because large proportions of Ipoth participants and non- 

^. participants obtained maximum scores, precluding thef'po'sstbility of significant ^ 

dif/erences between the groups.. Finally, Question^l yv^S <imitted since teacher 

re^sponses were used to answer tHe question kiid no^ corftpS^r^ons with non-participants 

wer^ p6ssible. U si rig the remaining outcome questions, the following results were 

obtaijied for each a/ t^ie/sarrvpling grpups: . ^ "I 
% ' ' y ' ' \* i • r 

'"'ill / / "'^ ^ ,^ - \ ^ ' ; *^ ' 

ts on 



**.\,^^/pra4e: Pa']^tj!^t|5ants scored more favorably tjian non-participsint 
• 'i/7(>;^vb.6tcome^qu^tlons: , ' ' . - •* 

'-^ - -/ ' ' ^^ //^ ' ' < : ' 

• 9th.(Sra6l^: Particip^jj/ir scored more favorably thcfij pon-participants on- 

2 of**^. o'utcome questidi/<^i " - * - ^ 

• .Parti^ipa'tw^. ^eacher Ord»ap {12th Grade): Pafticipamts scared mpr^ . 
^favorably £hk^' non-partVcipa^ts on 4 of 8 outcpme qu^^tions;/ 



< / 



Counseling Grduf>*(12th Grade),: ftarticipan^^s. scoredNmpre 
nop-particvpants- 3^o/^ outcoT^e .quVfi^tions/ ' \ '\ ' ^ v^- 



• Work.^^perience Group'/jf^KjGraj^^^^^'tibi^^^ scored more favorably 
than iion-partacipants^'6^ none pf the 8 putcome. questions'; and ' ; ^ 

/• Skill Training Groiip (U^,li-pra^e)( ^a^^^^^ "//^ 
r' ' than lion-participajits'^on one 'ofiirK^^8 OCCtcome questionaV i ^'*V^'I^"^ 
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TABLE 2.9 

Summary pi Conj^lusioiv for CJujcome Question^ Across aU Projects by Student Group * 



^ixp&nt partVcipants/^iil^ to idjenti(>9^reatey mrinber o£~ 
pations ' th an -non/ j/artic xpants ' (Q / ' / 

ity wita ta 



Are^sj 

c^cupatxons 



[ tasks and 



(artic xpants' (QJ-^ , 

I>5 stiiilents dcriorfsH^c©'im)re Camilia^i; 
functxqns asspciatc^'-^^rtH s^lei5ted o</cu'patioiis'than the /' , 
co«fparison'i^qU^?'''i[92) / ' ' / '/ 

i-/ ; ' ' 
Afc student r«nticipants,'xnpre familiar with the requisites ^ 

^ y ■associated witn «cfcctcd occupations tJian the comparispii* 



' * . ' * " stu^icil't' par ticipnnts sccw, jhigher pn pte -yboatioo^/, ;job 
' y * 'frp4<^j«ess tosts thnn.the'c6n\p^rison group 'otp4) 

Do stuJoht p>uticipants indicate more positiWattitudes 
tow ard 'cmplo>-ment than* non - participants .<(JS) — / 

Is the variety of careers being consider^ byjndividual^^ 
participating students greater than-that^f.studenif^^ -V ' - 
tlie comparison group'' (Qj6)* >^ , 

Do Qio^e',student participants, indicate having a career plan 
. than^t^^c coniparisQn^roup? (p8) / " ' ^ ' ... 



• V 



Do n^ore student partici points- cit^lthcir fcarcer preference as 
their expected^career'than non-p3rticiparits-?;(Q10) ^ ' 



Ves 



rV,es 

/ 

Ye»; 



^xes; 



No 



9th 
Grade 



Yes 



1^4 



Ko 



No' 



No 



Participating 
Teacher 



/Yes 



Yes 



> 'No 



Yes 



No 



12th Grade 



Counseling^ Wc^j^ 



Group 



Tel 



Yes 



No 



No 



No 



No, 



Y^s 



£xperien(se 



^o. 



No 



No 



No 



,No 



-No 



No 



Skill 
Ttainin'^ 



No 



No 



No 



No 



No'' 



No 



* The questions were afnsworea^ yes" tht^ce wa^ in overall significant difference in f^vor of partjcipants""/ (^'^lanv ofrthe • 
, , criteria used to assew the- qtrfistiori. ' \ *, ' ^ . ' " I 

'-'/ — '^r - A 's --V.- > — — — 



outcome qu^'ktiops >eiatej:b" the national objectives of the Part D Pf ograjp/aiTd^^^ 
as ^^sJeteritiii5S^4 -at tTie. outset of the s/udy, positive responses to these qu&^tVon^ v 
serve as indicators that, o*i^' a student leve.l, the program ha& succeeded in building^' 
'the bridge between school and the v^orld o^.work that. is required by the PartC^i'; 
legislation; ' 'Tie results given in^is^ chapter and summarized above seem t o ^ *' r 
indicate that'based on the analyses of all projects taken together, the program/' y' 
has succeeded in cettain areas and has hot achieved its student outcome objeptlves 
in othfeirs, , ' \ ' / • ^ 



7', 



/ 



It must be note'd that JJiese findings should'^not be interpreted & s a stCmmaary of all ./^ 
the student level outcomes alsaciated withj the progr^mi With. respect to the 
students enrolled in the Job Entry SJcill Training and Wb.rk Experience groups, for. 
example, this study does not^address the q|uestion pf the "extent to vjhich participants 
^were provided, technical skill* in the muUitude of po^^ble areas of training. 
. Similarly, sojne projects de^^oted con'siderable efforP^9^chieving objectifies which 
, ' were. np£ directly assocTat^^. with the outcomes measured and, of^coucse, an 

assessments of their Success \n attaining the se obj^cjfci ves was not made. 'As indica- 
ted at the!,fetoginning of this'chaptet, it should a,lso be noted that there is some 
^ in^ication^hat a se lection, factor which-\vas not anticipated by project staff resulted 
in, the participants^ in the Skill Traitiing ^nd \Vbrk licj>erience groups being somewhaj 
less academically oriented than'other participants^'^i^iheir comparison groups, 
; The finding's do, however, provide an in^i^ation of'^hfe extent to which project- 
supported activities made jscn impact tin Jiie studenfja ;^.hich .they served ir\ terms of 
th.e national program objectives,,. - - i'^t ' » 
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^Jjh'e results of the outcomp (Questions cin an individual project basis are summarized 
■ fff^.'t^bloS' 30 -'35. ' The se kble s show, 'for each sampling^^group, the number of 
p^pie^cts by the number*©f outcome c/uestions on which '^^a'^^ticipants scored signifi- 
cantly better tlian noh^p^^ti9ip^nts/ Four of the IS^oufcpme questions have, been 
elimiA^ted from these analyse'i.-^' These" Questions are 9, 11, 12, and 13. Question 
'7, vvhiclq'had* been^eliminate^ ^Tprp the^ovograll-surrinlary, 4ia5^b'een included, in the 
project-by^-project analysis koclsrijW^ £a .iV i^umUo^r^o^^ projects the*scor<is* 

were suffv,ci6ntly b^loAs^^ the maximum attt , i .- . . 

ticipants and non- participants. - ' ' 



/ 



1 t o \v arr a ht ^ia^-i:;t>n^^ '^^y^^ b ejfe w e qu pa r - 



Xhe results, frpm the project-by-projecJt^naJ.ysis were J[ess ?£vbi:at>ie--'43iajfi those 
^from^ tk^ acr-oss-proje'ct ai}ajysis. As ihdiclte<i-iQ th^ tables bel-ow, . a' relatively 
'small n'umbe.r Qf pr -vj^^ts^produc^d the favorable findin~gs' in each of the groups. In 
^no, ja^e did more thanr38^n of tbp pr'^j<*cts^ produce bi-Q or more favorable "outcomes. 



Nunr^r of projects by number of questions on which pa frtci pants scored significantly higher'than non -participants- 

6th Grade Groups ^ . ' \ ^ ' 



Number of Questions* 


6 I 


5 J 


4 


3 


2 • 


1 ' • 


.0 


.Total 


Number of Projects 

Percent of Proiecis , -^r-^v^s* 


. 0 
0 


\ 1 

2 


3 

7 


6 
13 


7 

15 


f . ' "T 

27' 


■ IG* 
36 


.45 
100 








' : . TABLE 31 
Number of ppojecb by number' of questions on which participants scored sjjuficantlv higher than non -participants 

9th Grade Group 

— ij , 


Number of Questions: 


7or ^ 
mdre ' ■ 




5 


4 


3 


2 


I 

1 


0 


Total 


Number of Projects 
Percent of Projects 


• 0 

' 0 


• 1 

2 


2 

5 • 


3 
7 


4 
^0 


6 
14 


11 

26 


15 
36 


42 
100 





• TABLE 32 

Number of projects by number of questions on which participants scored significanUy higher tfian non-participants 
\ ' ' ' 12th Grade Work Experience Group 




Number of Questions: 


4 or 
more 


_ 3 


2 


1 


0 


Total 




Number of Projects ' , ' 
Percent of ProHJcts • 


0 
0 


3 
16 


3 

16 


2 
10 


11. 
58' 


19 
100 ' 



•' * . TABLE 33 
Number of projects by number of questions on which participants scored significantly higher than non -participants ^ 

^ 12th Grade Skill Training Group 


' ^ . Number of Questions:' I 


2 or 
more 


1 


0 


Total. 


Number of fVojects " * 
Percent of Profects 


• .0 

0 


5 

f36 


9 
64 


14 

100 j:'". 



A... 



. * t -TABlE \ZA 

Number of projects by numbe/of questions on which partjcipajits scored significantly higher than non -participants 
' , ^ ^ ^2th Grade Participating Teacher Group 



N'l^ber of Questions: 


7 or 
more 


6 ' 


5 


4' 


3 


\ 2 


1 


d . 


Total ' 


Number, of Projects 
Percent of Protects 


0 
0 


^l 

4 


0 
0 


2 

9 


1 

.61- 4 


4 


8 
35 


10 
43 


23 
100 














\ 
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TABLE 35 

^ f Number of projects by number of question on which participants scored significantly hig*her than non -participants 

^12th Grade Counselor Group ' ^ 



Number of Questions: 



, Number of Projects 
Percent of Projects 



2\,r 



1 

25 



3 
75 



Tot^/ 



Only six of the nine outcome questions included in this analysis arc applicable at-tlie siicth grade level. 



tool / 



- 93 - 



7^ 




The Part D program seems to have made its^greatest impact on the 6th,. 9th, and 
12th grade Participating Teacher groups^- This suggests that both within projects 
and overall the greatest impact occurred through the infusion of career education 
concepts into the classroom by participating teachers. 

With respect to%vD x>f the outcome questions not inclyded on the preceding tables 
(i.e., "improved'l^ student behaviors, and placement aftBr'g'raduation), ' the over- 
all findings again are not completely clear. As indicated, placement data simply 
was not available apd therei9re no assessment of project impact could be made. 
Generally, the response^ of participatirikg teachers indicated they judged that the 
inclusion of career, education'activities' increased student interest in school and, 
to a lesser extent, decreased discipline problems, absenceis, and tardiness. 

Given the discus sioli in earlier chapters with respect to tHe activities .specified in 
the Part^D legislati|)n and U^OE policy paper which projects aadressed, the budget 
to. expenditure patterns which were followed, and the indications regarding pro- 
gram managemeat, , these mixed student outcome results were^to some extent to be 
'expuBcted. 

In the chapter v:^hicif follows we^iU investigate selected factors which may explain 
the variable nature^iof the outcomes which occurred. 



CHAP'^R DC: 



DISCUSSION AND INTERPRETATION OF STUDENT OUTCOMES 



Introduction ^ 

As reported in the preceding chapf'er, investigation of the indicatiors .of the student 
level outcomes* produced mixed r&ults. While all the fed^^rally expected outcomes 
did not occur as planned, tjiere wire several areas of student performance at each 
grade level wh^re^^y did occur .^Farther, the findin^s^in^Jicate that on a project 
by project basis tKe planned outcopfies occurred with sqprie uniformity in a small 
number projects, on a -highly variable basis in many projects, and in some 
projects rarely, if at all. 

As indicated in Chapter III, the sludy .design vide d for the collection of 'data 
pertaining to a set of research questions which were expected to be related tb tlie 
studen^ level results of the<-progra|n. Gi^^ that the basic purpose of the study is 
not only to report the results of th'e first years of the Part D program but also to 
provide information which may be of use in planning f^^ the future, the focus of 
tlie discussion in this chapter is on isolating those factors which may beet explain 
significant outco'mes; that is, particular attention throughout will be given to 
searching for those fattors which significantly contributed to the projects and 
program co*mpone;Jit^^ro(iucing the most favorable student outcome's. 

The findings below are divided into two major categories. In the first, findings 
and analysis v,ith, respect to the research questions identified prior to the data 
collection efforts are presented, 'The second section provides a-discussion of 
possible relatipaships between p^ogram^and project processes which were ^udgAd 
potentially important on the basiaof the findings reported in earlter chapters and 
other information obtained daring:^the vis'^ts to fifty Part D projects, 

f. > ^» Findings for JTreatment Research Questions 



This jfirst .category of fJossible explanations for the student outcomes addresses 
the s^^t of research questions agreed upon by the study team and USOE,in the course 
of developing the study design. The data upon which the answers to these questions 
are b^sed were obtained from the^randomly selecte<^ample of participating and 
non-participating students, and the random sample fl^eachers and counselors, 
JFor clarity pf presentation, the first three of the original set of six questions 
presented in Chapter III will be grouped and treated as essentially one ^Juestion 
with several' iiidicatots , 



1, RESEARCH QUESTION ONBf Are student participants exposed to more career 
familiarization curriculum activities than non-participants? 



A primary objective of the Part D program at the project' level was to stimulate 
teachers to increase the exti^nt to which their students were exposed to career 
related concepts and informlition. It was presumed that this basic objective 
would be achieved by alterii^ the classroom curriculum such that teachers 
would discuss careers and Specific jobs in cl'iss, would take students on field 
tripe vtjiere they would see people employed Mn the areas they were studying, 
and^would bring guest speakers into th6 clasproom to speak about their jobs and^^^ 
the r elationships'of their w^rk to the things |tht students were learning in class, ^ 
While it was 'found that in s<me casesv speci4l oJasses or designated segmerits 
^ of a class were devoted to studying a variet'j' qf^Jobs per se the typical approach 
Y was to integrate 'the study oi" occupations into on-going activites, • j,- ' 

To determine whether or not students participating in the Part D projects were 



exposed to more career far^iliarization cur|: 
pating students, data with r^pect to sever 



^jriculuA activities than nofx-partici- 
al criteria were collected and 



analyzedf* R^esponses of pa rticipants arid non-participants were analyzed 

both on ^ project by project basis and' across all projects. In terms of ' 

these indicators, participating students were judged to have been exposed 

to mor^ familiariz^jLon Activities when the chi-square test" to determine ' 

the significance of differences between two groups was significant at the 

5% levpl. In addition, ^e data were reviewed to determine if the 

expected relationship betwee-n career familiarization activities and the 

result's found for student outcome questions pertaining to occupational 

familiarization, job readiness, and altitudes toward work actually 

occurred. The findings for the criteria are presented separately below, 
i \ . , , 

Criterion la . All students were 'asked howx)ften their teacher talks ''abc»ut 
different k.inds of jobs. " Three response optibns were provided: 
"Hafrdly oyer, "Sometimes"' and "Very Often." This was question J6 of 
the//student questionnaire 

i\ - , 

Re/sults 



T^^ findings with regard to this criterion for each of the six student groups 
are shown in Table 36. The differences between participants and non-partici- 
pants were significant beyond the 1% confidence level for all groups except 
the counseling group of 12th graders, who, by definition, had no participating 
teachers. It is of interest to 'note that 75% or more of the participating 
students in every group excepd: counseling indicated that their teachers talked 
about jobs at least sometimes. ^ 

^ • ' A 

^ A summary of the distribution of the findings on a within-projects basis is ^ 
presented in Table 37. The results in twenty-five projects indicated that 
participating students received greater exposure than non-participants in at 
least one of the student groups. 

Criterion lb. AH students were asked if 'tl\ey had "gone on field trips this 
year to see people workings" Response options were: "No, " "Once or Twice, " 
and "More than Twice." This' was question #8 ^of the . student questionnaire. 

Results ' ► ^ . 



The findings for each group of students ar£ -shown in Table 36. Again, there ^ 
was a significant difference between the participants\id^the non-participdnts 
(beyond the 1% level) for every group but the iSth gr^^^i^^ounseling group, * . 
with participants' reporting m6re frequent field trips. It is of interest to note 
that SS% or more of the participants in each Ago up indicated that they had not 
gone on a single field trip to see people worl^g. 



A summary of the individual project data is shown in Table 38. The results 
in twenty-seven of tKe projects indicated that ^Participants engaged in field 
trip activities more frequently than*- did non-parti/;ipants in at least one of 



student groups » 



the 



\ 



Criterion Ic . All students^ we re asked whether visitors had come to their class 
"this ^ear" to talk about their work. The response options were: No, Once or 
Twice, and More than Twice. (Question #7, Student Questionnaire) 

Results 

The findings, shown in Table 36, indicate that there was a significant difference 
between the participants and non-participants .in three of the six groups: 9th 
graders, the 12th grade Work Experience group, and th^ 12th grade^ Participating 
Teacher group. At least 44% of each participating group reported that no 
visitors had come to their class during the year to talk about their work. 
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A summary of the individual project data \8 shown in Table 39, TheTesultf 

in twenty-four projects indicated jnore exposures tf visitors for .participants o 

than for non-participLants in at least one of the student groups, * ' 



TABLE 37 

SUMMARY OF WITHIN PROJECT FINDINGS 
FOR TRE/^TMENT QUESTION: 



"How often- does your teacher talk about different kinds of jobs? ' 




Student Group 



Number of projects with greater exposure for participants 
IVoJects without greater exposure for participaixts 

Toul Projects ^ ^ * 

% \y^th greater expovoie for participants 



6th 
drade — 


9th 
Grade 


121 


;h Grade Students 


Work 
Experience 


SldU* . 
Tralnlnji : 


Participating 
Teacher 


10 

35 


9 
33 


12 


5 
9 


" 18 


.45 
2296 


4i 
2196 


3796 


14 

3696 


2^ 
2296 



,* o ^ TABIjE 38 

SUMMARY OF WITHtN- PROJECT FINDINgS 
FOR TREATMENT QUESTION: 



'*Have yox gone on fJe'ld trips this year "to see/people working? ' 



. \ - 



Student Group 
Reporting Cateflror\' ^ 


6th 

Grade * 


9th 
drade 


121 


ii Grade Students ' 


Work 
Experience 


SkiU 
Training 


Participating 
Teacher 


Number of projects with greater exposure for participants 
Ptojccts without greater exposure for participants 

Total Projects 

% with greater exposure for participants 
* 


17'. 
28 


7 

35 


8 

- 11 


^\ 
12 


7- 
1^ 


. 4S 

3896 


42 

V7% 


19 
42% 


14 

* 

1496- 


/ 3096/.' 



•TABLE 39 ' ^ "^^ ' 

SUMMARY OF WITHIN RROJEC? FINDIl^GS ^ 'a. 
FOR TREATMENT fjUESTION:" • ^ 

»»Have visitor^ come to your class Ihis year to talk about their work? 



'* , Student Group 
Rcportine Cateijorv , 


6th 
Grade 


9th 
Grade 


12th Grade Stadtnts 


, Work 
Experience 


SkiU 
Training 


Participating 
Teacher 


Number of pici]^^^* with greater exposure for partiUpants 
IVoJccts without greater exposure for participants j 

Total Projects 

% with greater exposure for participants 


15 , ' 


12 
SO 


4>- 
IS 


2 

22 


8 

15 


45 


42 

2996 


19 

2196 


14 
1496 


• 23 
3596 
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Summary of Results ^ , 

I£ all three of the indicators of career familiarization curriculum activities 
are considered together', seven projects failed to shown any significant 
relation between treatment and participation and ten projects 
had a significant relationship tor each of the three indicators in at least 
one of the student groups. 

Relation.ship to student outcomes , ' 



In order to determine whether the obtained results on student outcomes 
could be rel^tec}, ^least partially, to the three treatment indicators per- ' 
tainihg to career famtH&riziation curriculum activities, the lists of projects 
in which there were significant outcome findings were compared with the 
lists of projects in which there were significant treatment criteria findings. 
The greater the number of projects which are on both lists, the^greater the 
relation between treatment and outcome. This analysis was carried out for 
each of the three treatment indicators and each of the eight outcome measures 
associated with outcome questions 1, 2, 7 and'lZ. »The results are shown 
in Table 40^ in termjs of: a) the rrumber of'^projects that had agreeniWnts -on 
outcomes an9 the treatment indicators; b) the number of such agreements 
that would be expected by chance (based on the total number of projects with 
significant relationships for outcomes and the total number, with significant 
relationships for treatment indicators), and c) the maximum number of*" 
agreements possible', given the number of significant relationships found. 
' Inspection of the tatJle indicates that relationships beyond chance were not 
found between treatment and combined outj:omes either b'^^^udent group 
or over all groups. Althaugh they are no^ shown in the table, the results 
are essentially the same for 6ach of the ei^ht outcomes taken separately. 

- The general conclusion regarding career fajmUarization curriculum activities 
is that participants tend to be exposed to ^hem^riore tl^n non-participants, 
but that this differential does not account for the outcome findings • 



TABLE 40 . * 

hiumber of Pj^ject Agreements Bedveen Having Significant Findings on CXrtcomes ' 
and Significant FindingsNsn Treatment Indicators 




Treatment Indicator ]la 
Number of agreements 
Chance expectations 
Maximum possible 

Treatment Indicator 
Number 'of agreements 
Chtmce Expectations * 
Maximum possible 

Treatment Indicator Ic 
' Num^r of agreements 
(^ance expectations 
, ^ Maximum possible 

Combined Indicators 
Number of agreements 
Chance expectations 
Maximum possible 



24' 
17 

68 



31 
30 
77 

26 
2'S 

77 

81 
72 
222 



16 
12 

59 

10 
12 
SI 

19 
19 

67 

% 

45 
43 
177 



7 
7 
21 



9 
9 
21 

2 
3 
20 

18 
19 
62 



0 
2 
7 

2 
2 
5 

4 
2 
5 

6 
6 
17 



3 
-8 
25 



7 
9 
29 

7 

10 
31 

17 
27 
85 



50 
46 
180 

59 
62 
183 

58 
200^ 

167 
167 
563 
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RESEARCH QUESTION TWO. Are more participating students assisted by 
school personnel in 'Securing jobs than non-participants^ ^ 

Criterion. In addition to compirying with the mandate that projects attempt 

to place all students in employment or further training after completion 9f 

their schooling, it was anticipated that many projects would provide assistance 

to students in finding. employment during the school year and during summer 

vacations.. As indicated in the previous chapter and in the discussion of 

project activities (Chapter VI), neither school nor project records permitted 

an adequate investigation of placement efforts. 

« , , «i 

% 

% 

Tlje only available data in this area was Questionnaire Item 39 which ' 
asked all 9th'and 12th grade students whether theXschooi has "ever helped 
you get a job?" jThe responses to the question provide some indication 
of the extent to which participating and non- pai^^ipating students 

^differed with respect to the amount of assist^nc^' they received. 
,^Since it was assummed at the start of the study that this difference, 

,;if it existed, might provide a partial explanation for the*di^ferences 
between participants and non-participants with respect to the outcome 
questions pertaining to Job readiness and attitudes toward work, the 
responses of participating and non-participating students at the 9th and 
12th grade were tested for differelices usihg the chi-square test. 

Rfes'ults and Conclusions * * 

' » 

The findings across all projects are presented .in Table 41. As indicated, a 
significantly greater proportion o^f participating students in the 12th grade 
Skill Traioing Groi^ and Participating. Teacher Group indicated that the school 
had helped them get a jpb. 

On a -project b^ project basis there was little difference between participants 
and non-participants with respect to this item. Specifically, there was a 
, significant difference between participants and non-participants m two 
projects at the ninth grade level. At the 12th grade level, significajit 
differences were obtained in three, two and one projects in tte 
Work Experience, Participating Teacher and Counseling groups,", 
respectively. These differences were all found in -different projects. 
While little difference was noted between the assistance provided participants 
and non-pkrticipants on a project by project basis,, the differences %at were 
obtained across all projects are of note. With respQ^ft to the Skill Training 
group, the data suggest that in addition to providing training the schools 
also offered more'assistance in securing employment to this group than 
otjier student groups. While *the students in the Participating Teacher 
group were not provided specific training in job entry skills throdgV the 
project, these students '^Iso appear ^o have been provided more assistance 
in securing employmei)t than non-participating students. While there was ' 
a significant difference found in these two groups between participants and 
non-participants, most students nevertheless responded negatively to the 
question. ^This finding, together wi^h the responses for the ather'groups ^ 
leSids to the conclusion that relatively few students in any.?gi?mip received 
assistance in securing a job. 

The results suggest that while the overall program may have had^some 
success in stimulating teachers and counselors to be of more assistance 
in helping students find employment, the impact has been marginal in 
each specific project ^e£ting. Although it could be concluded that project 
impact has been so pervasive throughout the school, thereby eliminating 
differences between participants and non-participants, the fact that the 

-•100- 
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project differences that have been found included instances where cont.rol 
group students were selected trom both within and outside of the project 
school s^ystem t^nds to s^ip^rt the int^xpretation of marginal impagt. 

As might be expected ffom^the relatively small proportion of student? who 
indicated they were^rovided assistance in securing a jobj and'^that 
variable/s general lack of relationship with participation, no. meaningful 
* relationship to student outcome question responses was found. 

3. RESEARCH QUtlSTION THREE. Do participating students receive mote 
occupational guidance and counseling during a school year than non - f 
• participants^? ! ' ' . - 

" Criteria . The Student Questionnaire forTfite 9^and 12th grades "Contained 
five items pertaining to this question, and ^^e Questionnaire for. the 12tb 
grade contained six additional items. These items are shown in the data ta\ljes. 
Comparisons werp made between 'participating and non-participating .students^' 
with respect to each of the items, both across all projects and within projects. 
These comparisons were evaluated by means of the cTii-squate test.' In the 
sections which folloy^-^iihe findings for all of the criteria will be presented 
first in terms of the across-project datd, followed by the within-project 
data. ' \ * 

) ^ 

Results Across Project^ I 

^) Items ^43 and #44 . The results for these two* queSti^s, as shown ^ * 
in Table 42, indicate no relation.between these indicators and participation, ^ 
^ except with regard to the counselors talking to individtial students about ho^* 

to prepare for their future at the 9th grade leveK It should be noted that / 
despite the significant relation with participation, only 33% of the 9th grade 
participants indi^^ed that they had received this treatment (compared to " • 
26% of the non~pa^cipants).j Approxirm tely half of the 12tl^ graders 
^ responded aff irrr^m|!trely . 

\ b) Items #45 and #46 . As shown in Table 43» -for J^ree of the five groups 

- there was a significant relation between participation and affirmative \ 

responses to talking with a teachc^r individually about the student's future* 
work and how to prepare for it. For each of the indicators, more 9th 
^ grade participants, more 12th grade Skill Training participants, and more i ' 
of the 12th grade Participating Teacher group responded affirmatively than. 
ihe corresponding non-participants. ^ -v - o 

c) It^m #49. Participants at the 9thgrade level reported' a greater . ^ 
. frequency of meeting with their guidance counselor than non-participants. 

The difference however, is -not great,, with 73% of the participants and 
68% of the non -participants indicating that they met with their counselor 
at least once. The only other significant relation found w&s for the Par- 
ticipating Teacher groOi^^^^e re 90% of'the participants and 83% the . 
n^-participants saw their counselor at least once during the year.^ These 
results are show^ in Table * * 

d) Items #50 a, b, c. The findings indicate that more Skill Training par- 
ticipants havp talked with a counsplor*" about getting a job ^nd about getting 
vocational or technical training than the nonrparticipants*. ^ As shown In 
Table 44, these were the only two instances of a positive relation between 
participation and the treatments. It is'of interest to note that the Skill 
Training group reported a smaller frequency of talking v/ath a counselor 

, about gettinga job than any of the other three participating groups, and a" 
> larger frequency of talking about getting further training. 
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e) Items ,^51 a, b, c . Tl)e finding for these items, shown in Table 45, 
^indicate tliat more participants than non-participanta in each 12th grade 
group except the Counseling group had talked with a teacher individually 
about getting a job (between 42% and 6l%)'and that more participants, 
, than non- participants in the Work Experience and ii} the Skill Tra jibing 
group had talked with a teacher about getting ^^o<;at:^onaV:br technical 
training .(43% and 41%, respeQtively);]^'^niy in the Ptil^'ticipating Teacher 
group di?i Vnore participants tV^Aa^^non-participants talk wjth a teacher 
about ap'plying to college' "(49%)* . .. , ^ 

Results Within Projects 

The number of projects in which a significant relation was found between 
participation and each of the 11 specific treatment items ia shown in 
Table 46. As indicated, , the frequencies are very small; in Qniy one 
instance does the humbe»r or projects in which there was a significant 
relation excefed 10% of thp /projects: 9th grade participants in 11 of the 
42 projects (26%) reported' meeting with their counselor; niore frequently 
Chan did nor^-participan;s. view ,r)f the small frequencies obtafned, 
analysis* tp- determine the relation bet\^''een these findings andfStudenfc 



oufcomes' \^as not warranted ^ 



C onclusion" 



\ 



I 




The. findings on this research question indicate ^that participating students 
receive more occupational guidance and counseling than non-participating 
students, but only to a small extent, and that on the 12th gr^de level it 
ap^^ears that such treatment curries more f rorrt teache rs than from counselor 

ESEARCH QUESTION FOUR, Do participating teachaiig encourage students 
to consider careers in vocational education? ^ ' * 

. : . . J . , 

/ ' ' I' 

' C rite rion . The ParJ: D legislation provides that pro^^cts should, among 
other efforts, "motivate and provide professional pj,efparation for potential 
teachers of vocational education, " To obtain an inaicatlon Of the extent 
to which this occurred and rts relationship to .the student. respons'es with ^ 
regard to vocational education as a career choice, the participating teachers 
in each project were a^lked: "Do you encourage students to go into voc.a- 
tibnal or technical' t eaelhing (Teacher Questionnaire Item 10)^.* 
Response options wej*^':. "rarely or never, ". "sometimes, " and "often. " 

Resialts and Cdnclusion . In Chapter VIII it was reported that, riat\onwide, 
only 12 students cited vocational education as their futuref caree r. 
Howev**r, when teachers were asked whether they encouraged students 
to go into vocational tf* aching, 32% of the participating high school ^eachers 
in^fcated that they often did so; 48% indicated that they did so "sometimes"; 
: an<5 21% responded "rarely or never.*" From these rfe^ults it is apparent ■ 

that,^ in this specific instance, »teache.rs were not able^to influence their 
' students, to any degree. ^ . ^' ' 

C. Other Potential Relationships ' 

'' ' V ■ V' 

In the process of searching for explanations of th^ variations in student ioutcom6s 
acijps^s student groups and across projects, the full range of information obtained 
during the visits to the fifty projects' was analyzed. ,This included information pe,^- 
ta irking to project staffing patterns, organizational structures, stated objectives, ♦ 
and other* factoids which were discussed in earlier chapters of this report^. Patterns 
i^/h^ch distinguished projects on the basis of the student outcome questions. idicf ttpt ,^ 
'e'm^rge.,as a result of this review and analysis. . w ,\ 

' - V , ' ' , V ^' 
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^ . ' TABLE 45, . 
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\ \\ NO 
\^\^ Totals 


277 
178 


GO 9 
5D I 


2H 
201 j 


51 6 
4S 4 


ISS 
214 


42 5 

57 5 


133 
?83 


32 0 
68.0 


276 
348 


44 2 
55 8 


215 
455 


32 1 
67 9 


29 
65 


30.9 
69 1 


51 
124 


29.1 
70 9 


4SS 


100 0 


415- 


100 0 


372 


100 0 


416 


100,0 


624 


100,0 


«|0 


100 0 


94 


100.0 


175 


100.0 


\\ Sn^naicance 


(pill 09) 


(phi - 13) 


(pJ.i - .13) ^ 






ns 




vS)li> Hjs ^ uVciter t <lked 4bout Jpply- 
in;; to «>^fiC ' 

T 


2^7 


46.9 
S3 I 


20! 
212 


► ;s 7 


212 


41 3 

58 7 


191 
226 


45.8 
54 2 


305 
319 


48 9 
51 1 


287 
380 


43.0 
57 b 


33 
60 


35.5 
64.5 


75 
93 


43.4 
56 6 


446 


100 0 


413 


ltX).0 


3t>l 


100 0 


417 


100 0 


624 


100 0 


667 


100.0 


93 


100 0 


173 


100,0 


v.. 








(phi - . 06) 


ns 






iSlc Hav a teacher taU<e(Jwbtoiit RCltir^ 
vocational or technical iVining ? 


189 
25S 


43. J 

57 7 


149 
26S 


36 0 
64 0 


150 
212 


41 4 

58 6 


103 
309 


25 0 

. 75 0 


198 
425. 


31.8 
63,2 


185 
479 


27 9 
72,1 


20 
73 


21 5 
78 5 


41 
129 


24 1 
75,9 


447 


100 0 




100.0 


362 


100 0 


ii2 


100,0 


623 


100 0 


664 


100,0 


93 


100.0 


170 


100 0 


\^*^-^ Sf^nificance n ^ 
~ -< r* ' ■ ■ ?T 1 
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• -•^»>ififf*ot^^05 level of confidence 
*• - jl[^lfi<;«m"at-'-01 level of confidence 
nj = nol^^ftl^'anP^ 
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"r^r, Treatmpnt and P.'^rticip,itxon 
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In addition to this analysis, the responses to the questions asked of participating 
teachers and counselors and of non-participating teacher^ were analyzed. The ^ 
results of this analyjsis also did not produce patterns which served to explain tl^e 
student outcomes reported in Chapter VIJI. It was found, fbr example, that bpth 
across projects and on a project-by-project basis most counselors did not indicate 
they had extensive experience outside the school system which would assist them 
m placing students in jobs. While this corresponds logically with the stud^ent 
responses ind<catir>g that they received relatively little assistance from school' 
personnel in securing employment, there was no relationship between the projects 
in which several of the counselors indicated that they did' have such out-of-school 
experience and those in which significantly more participating than non-participating 
students indicated that they had been assisted by sch6bl personnel or those in 
which participants uiJlcated more favorable attitudes toward work. » 

1 , . 

Similarly, results of comparing counselor and teacher responses with respect 
to the extent of in- servrce training in career education to various . 

student. outcome indicators did not provide explanations for the variations in project 
outcomes.^ In fact, 'the results of comparing responses of participating and non- 
pa rticipatpg 6th grade teachers suggested that, in tern\a of the use of community 
resoar^ces, in-service training in career education, the importance of including 
career education in the school curriculum, and roost other responses which might 
^ be related to career familiarization outcomes, there was surprisingly little differ- 
ence between the two groups of respondents. • 
* ' \ A 

There was a significant difference, however, on one indicator whi^h may at least 
suggest an explanation for some of the difference between participating and non- 
participating 6th graclers across projects. Both participating and non-participating 
teachers were asked^ whether they "have a written curriculum guide for including ' 
career education in \rbur classes ?" In response to this item 22% of the non- 
participating as compared to 62% of the participating teachers indicated that they 
had prepared such a.^uide. 'While on the one hajid the difference between the two 
.groups in the response to this item may be related to the generally positive out- 
. comes of participating 6th graders, the fact that 22% of the non-participating 
teachers also had a curriculum guide "for including career education" in their 
classes is perhaps the more important finding. It suggests that in some cases th^ 
lackjof observed differences may have been the result of eff6rts on the part of * 
both,,state and prbj<*ct staff to persuade teachers ifot participating in a Part D 
^ -project to infuse career education concept's into t^eir curricula. Given the 
' ^ pattern which did emerge, however, thete is some indication that other factors 
may have been operating. 

Since neither the analysis of specific responses from teachers and counselors nor 
the analysis of various factors associated with project management and operations 
provided meaningful explanations for the findings, an additional analysis was^ 
caJ^r ie d out in which projects containing the bulk of the positive findings were com- 
. pared \o all project^, in terms of the number pf students served in each group and 
the changes in these numbei;^ over time. This analysis is described below. 



As indicated previously, the answers to several questions w^re to ^erve as 
indicatars of the ex^nt to which student outcomes consistent with Part D legislative 
intent were produced. Six questions served as indicators at the 6th grade level 
and nine served at the other levels. For purposes of this analysis answers to a 
questioh were considered favorable with regard to the Part D legislative intent if 
for at least one indicator 'of that question participants did significantly better than 
no n -participants. The i^^'S'ults of this analysis for each student group are discussed 
in the following sections. * * , ^ 
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1. Elementary Level 



At the sixth grade level, student data comparing participating to nonaiparticipating 
students were obtained in fo|rty-five projects. Table 47 indicates tWe number of 
pi-ojects in which different pumbei's of outcome questions showed-more favorable 
responses the participants. From the table, it can be seen, fbr eickmple, ^that 
in one project participants clid significantly bettjsr than non-participarits in five out 
of the six questions. ^ « <^ ^ ^ 1 



TABI.E 47 

NUMBER OF PROJECTS BY NUMBiR OF OUTCOME QUESTIONS ON WHICH 
SIXTH GRADE PARTICIPANTS SCORED SIGNIFICANTLY BETTER 
THAN NON-PART ICIPANtS 







* Numbe r 


Perceift 


Percent of- 






of 


of 


FaV o rable 


Number 


of Questions 


Projects 


Projects 


Answers 


« 


0 


16* 


36% 






i. 


12 


27% 


2iD% 




Z 


7 


* 16% ' 


23% 




3 \ 


. 6 


13% 


30% 




4 


Z 


7%^ 


20% 




5 i 


1 


2% 


'8% 






_0 


_0% 




total 




45 


100% 


100% 



* In 2 projects the participates and non -participants sampled were not -equivalent in eth^c compositioT). 



T 



As: Table 47^ Siows, there were no, positive student outcomes in 36% of the - 
^pj'ects,' Participants did better than non -participant s On three or more of 
the "study questions^'in te^ or 22% of the projects 



for S8% of the favorable answers found. 



These projects accounted 
For purposes of analysis the average 



number &f elementary school participants in these ten projects was compared 
to the ^average number of elementary school participants in.the total grWp of^ 
, 45 projects, It^was fovmd that for "each year the average number of students 

substantially lower for 4he 10 projects than for the total group of projects. 
As shown in T^ble 48, during the first project year these ten projects renorted 
a per project ave rage of participants that was 19%'below the average jfor the 
total group. They were 26% and 22% below the total group's average in the 
second and third year, respectively. 
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1 • TABLF 48 

! COMPARIS(^isI.OF AVERAGE NUMBER OF SIXTH GRADE PARTICIPANTS / 
, _v PER PROJECT TO THE AVERAGE FOR THE 10 MOST FAVORABLE . / 
' . ' PROJECTS, BY PROGRAM YEAR - ^ 




Average Enrollment 


PY I PY II PY in 


AH p^rojects: 
Average Number of Elementary 
1 Participants /P roject 


1709 1977 2512 


10 Most Favorable Projects; 
Average Number of ^Elementary 
Participants /Project 


1390 1466 1958 


1 »■ Diffe rence 


' 319 511 554 


vPf^rreat Fewer Participants in 
[10 Most Favorable Projects 


'i9% zl% 22% 



4 . 

2. . Junior High School Level 

Test results were obtained m 42 projects at the juhior high school level. As 
indicated previously, there were nine resea rch questions serving as indicators 
of student outcomes at this level. The result of com'paring participating 
students to non-participating students (shown in Table 49) reveals that in 15 of 
the*42 projects (36%) there were no positive outcomes. Participants scored 
higher than non-participants on three or more of thfe research questions in 10 
projects (24%),^ This accounts for 64% of the favorable ^swers found. Com- 
parison of the average aumber of junior high sch(yol participants in the ten 
most favorable projects with the average reported for all projects, as 
Vhown in Table 50, reveals that the average for the 10 friost favorable projects - 
was substantially lower than the average for, all projects. The 10 most favorable 
projects averaged 6l% fewer j\mior high participants yei* project than did all 
projerts the first year. For the second and third year these 10 projects were 
577o^nd 507o belpw the average reported for all p'rojects. V 

- . / , • - ■ 





TABLE 49 






NUMBER^OF PROJECTS BY NUMBER OF OUTCOME QUESTIONS ON WHICH 
NINTH GRADE P^RTICIPANTS^SCORED SIGNIHCANTLY BETTER 
'THAN NjON-PARTICIPANTS 


Number of Questions 


i« ^Number 
of 

Projects 


Percent 

of " ' 
Projects 


Percent of 
. Favorable 
Answers 


0 ' 

1 * ■ 

3 

^ . • 4 

. 5 
6 

7 or moie' 

Total 


11 

6 

-C r 

0 
42 


36% 
26% 
14% • 
10% 

7% * 
'.5% 

2% 

0% 
100% 


17% 

19% 

19% 

19% 

16%^ 

10%' ' 

ioo"% 



* In one project the participants and non-participants were not equivalent in ethnic composition 



TABLE 50 . . • ^ 

^ COMPARISON OF AVERAGE NUMBER OfWh GRADE PARTICIPANTS 
PER PROJECT TO THE AVERAGE OF THE 10 MOST FAVORABLE 

PROJECTS, BY PROGRAM YEAR ' \ 




Average Enxollment ^ 


PY I PY n ^ PY m 


All Projects: 
Average Number of Junior 
High ParHripants /Project * 


' 1215 1416 ^ 1611 


10 Mt>st Favorable Projects: 
Average Number of Junior High 
Participants /Project j 

~ ZI— v-^!: 


474 612 ' 806 


Difference 


■ ■ ; ■ 

741 894 805' 


Percent Fewer Participants in 
^ 10 Most .P^v.orable Projects 


61% 57% 50% 



3. Senior High Level 

a) Work Experience Group * 

At the senior high school level there weVe 19 projects for which comparison^ 
between participating Work E*:perience students and non-participating Work 
Experience students were made. Differences between participants and hon- 
participants were observed on seven outcome .questions. As can be seen in 
Table 51^ seven, or 37% of the projects accounted for all of the favorable 
answers found. 



TABLE 5i 

% > 
NUMBER 9F PROJECTS BY NUMBER OF OUTCOME QUESTIONS'^ON WHICH 
^ 12TH GRADE WORK EXPERIENCE PARTICIPANTS SOCRED » 
SIGNIFICAI^TLY BETTER THAN NOr4-PARTICIPANTS 




Numbe r 


Pe rcent 


' Percent of 




of 


of 


Favorable 


Number oi Questi6ns 


P rejects 


Projects 


Answe rs 


1 

* 2 

3 . 

4 or more 

Total ^ ^ . 


12 
1 

3 

\ 3 
' JDI 

19 


63% 
5% 

-1-6%-^. 
16% 

-1 ■ 

100% 


6% 
38% 

1 56% ^ 

100% 



jii - 



T 



These seven projects reported an average number of ^rtjLclpants per project" 
that was smaller than the average for all projects. This Tl^ shown in Table 52. 
An average of 8% fewer participants in the 7 most favorable projects than in 
all projects was fpund in the first year. There were 15% fewer in the second 
year, and ^1% 'fewer in the third year. ^ ^ 
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TABLE 52 

COMPARISON OF AVERAGE NUMBER OF 12T|i GRADE WORK EXPERIENCE 
PER PROJECT TO TWG AVERAGE FOR THE 7 MOST FAVORABL? 

PROJECTS, BY PROGRAM YEAR ^ 



All Projects: 
Average Number of Work 
Experience Participants/ 
Project 



Most Favorable Projects: 
Average Number of Work 
Experience "Participants / 
I^oj e c t 2^ 



Difference 



Percent Fewe r , Participants 
in 7 Most Favorable Projects 



Average Enrollment 



PY I 



PY II 



PY m 



84 



110 



128 



/ 77 



94 



101 



16^ 



. 27 



8% 



IS? 



21% 





b) Skill Training' Group ^ \^ 

Of the 14 projferts for which comparison data were available,- skill training 
participants sc^ed better rthan non-participants on one question in six projects. 
In eight projects Skill Training participants did not do bette;r than non-par tic ip ants 
on any outcome variable. /These data are shown in Table^53. 
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/ ^ TABLE 53 ^ ^ 








^NUMBER OF PRO! 
12th GRADE SKlZ 


a > 

PCTS BY NUMBER OF OUTCOMi; QUESTIONS IN WHICH 
,U#rRAil^ING PARTICIPANTS SCORED StpNIFICANTLY 
BETTEyR, THAN-NON-PARTICIFANTS 




Number of Questions 


Number 
of 

Projects 


Percent 
of 

Projects 


Percent of 
Favorable 
Answers 




^ \ 

0 I 

1 I 

2 or more \ 


- ^ 8 ' 

6 

L ■ * — 


57% 

. 43% 
Jl% 


100% 
0% 




Total 


\ ' ' ' 14 


100% 


100% 




■t 


. 112 


* 








* 


4 


1 , « 





A comparison of the average number of Skill Training participants per projett 
in those six proj^ts where a favorable answer was fotind with all projects 
reveals a considerable difference. As shown in Table 54, in those projects ' 
with a favorable answer/the average number of participants per project is 
much higher than the overall average of Skill Training participants per project, 



/ 
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TABLE 54 



COMPARJLSON OF AVERAGE NUMBER OF l^TH GRASQEJSKILL TRAINING 
PARTICIPANTS PER^ PROJECT TO THE AVERAGE ^R THE 
6 MOST FA\^ORAB-L>E PROJECTS', BY PROGRAM YEAR 



All Projects: ' 
Average Number of ^kill ' " 
Training Participants /Project 



Average JMumber of Skill Training 
Participants for 6 Projects with 
Favorable Findings ^ 



T 

Differencfe 



Percent More Participants in 
Favorable Projects 



Jlnrollnnent 



PY.I 



PY II 



338 



408 



564 



740, 



226 



332 



67% 



81% 



PY III 



355^ 



73 5 



380 



107% 



The six favorable projects averaged 67% more Skjlll Training participants per 
project than the overall project average for the first year, T^e difference / ' 
between these six and overall project averages increased each program year 
with the six projects having an 81 %« higher average the second year and a 'i07% 
highe r average the third year. This enrollment trend is even more striking 
in view of the fact that for the three years the average number of Skill Trailing 
participants over ^11 projects increased only 5%, while for these six projects 
the number of such participants increased by 30% during the 3ame thre6-year ' 
period, - * 

c^) Participating Teacher Group ' t » * 

There were 23 projects for which comparison data between the 12th grade 
Participating Teacher group participants and non-participants were reported, ' 
Table 55 indicates the number of projects by number of outcome questions on 
which participants scored significantly be^tter than non-pfirticipants« 
Participants did significantly better than non-partifcipants with 
respect to one or more of the outcome questions in 13 projects, or 57% of the 
total. In addition, five of the projects {22^o) contained 70% oi the favorable ^ 
findings, /' * 
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TABL£ 55 






NUMBER OF PROJECTS BY NUMBER OF OUTCOME QUESTIONS IN WHICH 
12TH GRADE PARTICIPATING TEACHER GROUP SCORED 
SIGNIFICANTLY BETTER THAN NON-PARTtCIPANTS 


/ Nuriiber 
f of 
Number of Questions P.rojects 


Pe rcent 
of 

Projects 


Percent of 
Favo rable 
Answe rs 


0 . 10 

1 ' 8 

2 1 

3 • I 

4 ,2 

5 . 0 

6 • . 1 

7 or more ' 0 


43% 
. 35% 
4% 
4% 
, 9% 

* 4% 

0% 


30% 
.7% 
11% 
30% 

2?% 


Total 23 


100% 


100% 



The average number of stud-ents in the 12th grade Participating Teacher proup 
did not noticeably incr'^^oe or decrease during the three year terryf of^art D 
program support. However, for j;he five most favorable projects the average 
number of students per project increasecf 153% ov^er the three year period. As 
shown in Table 56, the average number of participants in these projects was 61%, 
23%', and 1% fewer than the average for all projects in year I, II, and III,' 
respectively. . , . 



A 



TABLE 56 

• « 

COMPARISON OF AVERAGE NUMBER OF 12TH GRADE PARTICIPATING 
^ , TEACHER PARTICIPANTS PER PROJECT TO THE AVERAGE ^OR 
^5 MOST FAVORABLE PROJECTS, BY PROGRAM YEAR 


Reporting Category 


Average Enrollment 


PY I PY II , PY m 


All Projects:. ' v 
Avet-age Number of Familiarization 
Particfipants /Project 


675 ^685 . . 673 


Average Number of Familiarization 
Participants for 5 Most Favorable 
Projects ^ 


263 ^ 528 ' 666 


Difference 


; 413'"^ .,157 " ' I / 7 


Percent Fewer Participants in Most ^ 
Favorable Projects 


h • . • / 

^ 61% 23% 1% 
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The five projects reflect a pattern^ of student participation which is sigii^ar to 
the ove rail patte rn found at the elementary, and jundor high school levels, . This, 
however, Ts no* the case f,or the total group of projects having" IZth grade 
sLuuents in the Participating Teacher group, * 

d) Counseling Group ' 

' * 

There was one project in which 12th ^rade counseling group participants did 
better with^Xespect to one re'search question. Analysis produced no signifi- 
cant insights with respecf to this area, ^ 



• D. Summagy^and Conclusions » 

The findings presented relative to the program treatrr\ent questions estab- 
lished at the stfart of the study indicate that, with 3:pspect to the overall analyses of 
projects, projects tended to provide participating students with significantly more 
classroom experiences conceptually associated witk career education and occupa- 
,tlonal familiariz;^tion student outcomes than were provided non-participants. In 
practice, howevefr, no relationship was foxing beyond the levdif of chance between 
student responses regarding these treatments arid student outcome results. Similarly^ 
no relationship between response**? of coun^^ors and teachers and student outcomes 
presumed to be associated with these respjo^^s could be clea?ly established. 

The search for explanations did,^ howeve r, reveal that a relationship exists 
between the scope of the projects in terms ^of number of participants and meabures 
of student outcomes. Specifically, 

• at the 6th grade and. at the 9th grade, the average number of students served , 
by the 10 projects with the. greatest nurnber of favorable student outcomes 

was less than the average number served by all projects; 

I 

• for the Work Experience group, the average number of students served by 
^ the seven projects with significant outcomes was less than the average 

number served by all projects; >^ 

^ 5 for the 3rvill Training group, the average number of^students served by the 
six projects^with significant outcomes was considerably greater than the 
average number served by all projects; and 

• for the Participating Tpacher group the average number of students served 
by the five projects with the greatest number of fav curable student outcomes 
increased ISSfo from Year I to Year III, compared w^th no net change in the 

^ average number served by all projects. * 



These findings will be related to cost information in the next chapter in an effort to 
further explain the results. , ■ ■ • 
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CHAPTER X: COST ANALYSIS 
^ • 

A, ^troduction 

In Chapter V, project budget and expenditure inforniafcf*^ was used to describe and 
compare the planned federal inputs to the inputs that acfucfclly occurred during the * 
three-year tern^ of the first round of the Part D program, From^that data it was 
concluded that for most projects the inputs did not occur at the levpl planned, ' ^ 

Another \ise of cost data is as an indicator of effectiveness or efficiency when com- 
paring programs or performance over time, Jn this chapter] ad4itionaI cost data 
is presented and analyzed in an effort to develop further insights relative to program 
efficiency and effectiveness, ^ 

In recent years cost analysis techhiques, especially with regard to ^ucational . 
programs, have beconne increasingly sophisticated. They all haVe as their purpose, 
however, providing data which permits managers, planners, and policy makers to 
compeire alternatives and make decisions concerning the best use of 
resources ba'sed upon past performance^ The following discussion does not rely 
on anyone techni4jie of cost analysis i cost-effectiveness, or cost -benefit analysis^, 
Ratfier, it is an attempt to provide, for management, planning, or policy making 
purposes* useful information concerning the three-year performance of Part D 
projects with regard to program costs. 



Attempts to secure us-eful cost data during the field tests of instruments and proCe- ^ 
dures revealed that adequate cost ddtd were ditticult to acquire in many instances, * 
Several factors were involved: % * . * 

# 

• Many projects did not use budget-expenditure data as a management tool 
, . on a regular basis, '^This resulted in program staff frequently being un- 
familiar with budget^ and expenditure issifies, 

• Grantees usually were not able to break out the costs of a given activity, 

• Grantees frequently had difficulty describing funds from local sources that 
4 we^re applied to support project activities, 

• Grantees frequently could not clearly describe project activities or 
participants, * ^ 

Because of these problems, cost data were sought in two simplified forme* Budget and 
expetiditruic data were sought according to the categories contained in the USOE 
Sample Budget contained in the publication of Instructions and Procedures for - 
Proposing and Conducting Part D Exemplar?/ Programs, It was fejt that this type 
of data would be relatively xaniform across projects and easy to obtain gipce this 7 
was the format utilized by^the grantee in requesting funds and by USOE ^in greeting » 
them. It was expected that most grantees 'either would have information in this 
form or , could .develop it in this form with little difficulty, 'Sttfnmaries of this infor- 
*matiQn were presented in Chapter V, In addition, in order to make comparisons 
across projects, it^as decided to attempt to secure expenditure information by fcne 
seven activity areas being studied. 



As noted earlier, there was wide variation in activities from one pi*oject to another 
and definitions of activities and /or participants frequently were qv^te vague. Based 
upon tne results of. the field tests, it was concluded that activity cost data w^uld, of ' 
necessi^, be based on estimates provided by project and grantee staft^ ^Xirther, • 
it was concluded that the level of detail sought with^espect to activity costs would ' - 
have \o be limited to two broad categories: personnel costs and .other costs. These 
data were sought for each year of project operation on a program year basis... As 
expected, the projects spent tne great bulk of the Part D funds for personnel. 
Since this pattern wajs similar for all projects, this chapter largely focuses on data 
far the activity areas utilizing total costs within each activity area. 

B. Findings; . Cost in Relation to Project Activities 

Overall budget and expenditure data were^repoifted in Chapter V (see T-able 7), In 
brief, that Chapter pointed out that 39 projects reported that they underspent their' 
first year budgets by an average of 19. 2% per ijroject. For the second year, 38 
projects reported an average underexpenditure'of 16. 0%, and for the final program 
year the underexpenditure averaged 12. 1% for the 1% underexpending projects. 

\* * 
In addition to overall budget and expenditure da^, projects were aske<J to provide 
an estimate on the cost of project activities in ^rms of the seven activi^ty areas 
outlined in the legislation. Forty-three of the fifty projects studied (8S%) provided 
these estimates. . . ' ^ 

The estimates are presented in "fable 57 by activity iyrea for each of the three years 
of' Part D funding support. Activi.ty costs are expressed as a percentage of the 
total Part D expenditures for the program year. This presentation format was 
adopted wh^n review and analysis revealed that ipany projects were not able to 
provide dollar estimates. ^ ^ ^ • ^ 

1. Elementary Activities (Grades 1-6) 

r i 

Thirty-eight projects reported elementary activity expenditures in their first 
jirogram year. Tkese projects reported an average percentage equaling 28. 
of their Par t-Ef expenditures for these activities. The 40 projects reporting * 
. elementary a^ctivities expenditures in the sfecond year spent an average per- 
centag-e.of 24.^8%. ^ In the third year, the 36 projects reporting elementary, 
expenditures re^^Trted spending an average percentage of 24. 5% of Part D 
expenditures, on activities at tnis level. * * . " 

, Four of fortyi-three projects p^roviding activity cost estimates for the first ^ 
program year reportej no expenditure of Part D monies for elernentary activi«r 
ties. Of these ^our^, one reported second year expenditures for j^etnentary 
activities and one reported such expenditures for the third progrjfen year. *The 
remaining two .reported no elementary school activities during t)ne three-year 
term of Part D funding. One project reported expenditures for elementary 
" activities during the first two program years but not the third. It was the only 
reporting project where expenditures were terminated' once initiated. - 

Table 58, page 121 , compares the average percent of the total Part E[^project 
^cos't' reported for elementary school, activities by program year and the average 
numbeV of elementary school participants per project. Seven of the fifty projects 
did not report expenditure data by activity area. An inspection of the number of 
participants, total anny^al budget and expendjlture data, 'and other available 
information indicate that the averages presented here are an accurate reflection 
of average proportional project cost for these activities. 

' ' . 

* The lower number of projects reporting elementary activity expenditures in the third year is attribute ^o fewer proj^ctp 
reporting fpr this year father than a- lower Ic^l of elementary activity. 
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■ // TABLE 58 
CO.MPA RfSDN OF /ELEMENTARY ACTIVITIES VERSUS COSTS 



/ ' , ' 


Program 
" Ye^r I 


Program 
Year II 


' Program 
',YeaiM!i 


Average Percent of Total Pact D 
Projects Cb5j5 i 

Average Nuipber of fclcmentary 
Participant*; per Project 


28.8 '--^ 


, . 24 8 
1.977 





» ^ As, indicated in^tlfe table, dxjxi'rig the thir^-y^^p rejects reported that 
oi tVie Pai^ D'^unds we,3?e "devoted to^elementaiyy school activities, *On 
^-^^and^^^Iementa^ry.s'chool part^ accounted for 51. 9% of the tti^l 

"i ^-of iPart D.parficipan;;s in- clll projects^ ... 



24, ,5% 
the 'other 
rtuuibe r 



^ .Six proj^t^^ jp^orteid elementary school activity expenditures d\^ring,the first 
• *B5<^i?^^^1^^ar arvdiHO elementary school participants for that year. This may be 
explained, in ^^art, by the nature of the start-up activities employed by the 
projects. These projects reported that the first program year -was devoted to 
c^lirriculum.'deyelopment, material development, and staff (development and 
training. One project reported that high initial staff turnover was als^o a 
. f>^ctor in the lack of student involvement the first year. ' ^ 

^ Table 58 clea;'ly ^hows that with each year proportional costs decreased \yhile 

'participatnts increased. Part of the increase in participants is accounted' for, / 

^^h<f p/ojects\\iiichliad previously reported expenditures for planning and stari~up 

}>urposes /an4'were now reporting participants. This }s. particularly true .for ^ 

Ihe first/year/secohd year figures. The third year .figyres, on the other -hand", 

appear to reflect greater efficiency in the delivery ai'program services. - 
• ' * ^/ 

Junior High School Activities (Grades 7-9) / . '* //- / 



/ 



/ 



/ 



From Table 57, it can be seen that 34 projects,vrepbrted first year costs for 
juniorhigh schooractivities.' The average perc^tWge^spent on these activities for th 
reporting was 29. 3% of the total Part.D proj^c't/^ojst. Thirty-nine projects ^ . 
reported second War junior high school actiyi^fgs at an ave'rage 
percentage of 31^0%. For the final yjearl;'^''6 projects reported expending 

*an average of 27'. 6% oft^their .total project cost'a/the junior high school level 
while serving 33. 5% of all third year prdject/pVl^ic'ipants. * 

Eight^of the projects providing activity/cost .estimates indicated that no-funds 
were devoted to junior high school activitiej^g the firs^ year. Three of these 
did not indicate expenditures/during the s<^cond yeaV. One projejLt indicated 
expenditures in this activity area, for the tinal'year only, anli tw4 projects 
reflect no junior^high activity costs for all three yea^rs. One^oject repo 
costs only in the second year. ' - r 



ported 



Table 59 compare s"^e average cost of jtinior high school activities expressed 
as'a percent" of the total'project co^t (Part D funds only) to the average number 
of junior high school participants for each'yfear of Part D support ' 



. 
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TABLE 59 

HIGH SCHOOL ACTIVITIES VERSUS (^OSTs' 





Program 
• Year I 


Program 
.Year II 


Program 
Year III 


Average Percent of Total Part Project Costs- 
Average Number of Participants per Project 


29. 3% 
1,215 


31.094 
1,416- 


27.6% 
1,^11 



As was the case with 



elemen|:ary activitie^,;jthet*e was an increase in the average 
numbef of participants p'er '-^rcJject f c3^|e ajjh, of the three years of Pare D support. 
Proportional costs did increase slight/y i4 ,ti^e second year but in the third'year 
decre^^sed below the first year level. Thj!6;^pattern also appears to reflect the 
'-j^effect of both planning and start-up costs^lnVearly years and program efficiency * 
in* the thi^d year. - v '* ; X 



3. Senior High School Activities (Grades 10 v»12) 

Of the 43 projects reporting' cost estimate!^ 39 reported first year senior 
high school activity cost?s. Table 60, helow, indicates the average Part D 
cost of all high school activities as a percent of the total Part D project 
cost. The average number of higli school partic^ipants per project is also 
shown in the table. _ 1 



^ TABLE^O , 

COMPARISPN OF THE TOTAL SENIOR mcK SCHOOL ACTIVITIES VERSUS COSTS 





Program 
Year I 


Ptogram 
Year II 


Program 
Year III 


Average Percent of^TctalPart D Project Costs 
Average Number of Participants per Project 


42.1% 
552 


44 29^ 
668 


" 47.996 
723 



i 



Senior high participants accounted for 14. 7% of the total third year Part D 
project participants. However, senior high school activities were Reported 
to have accounted for 47.8% of the total Part D project costs. 

Unlike the elementary and junior high levels, both the percentage of costs and 
participants increased yearly for the hi^ schoolUevel. Thus, it appe,ars that 
high school activities involved relatively fewer planning and start-up costs and 
the' addition of new participants was fully funded. In addition, it wniild appear 
that program efficiencies did not occur at the liigh school level. 

^ These data would suggest that in general the^three year experience with 
respect to Elementary level' programr^ins'and junior high level programming 
wasj similar and that both^ varied considerably from the overall three year 
experience at the high school level. The^' components of the high school 
Effort are discussed on the following page. 

■V 

i 
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a) High Schoi)! Work Experience Activities ~' " 

High school participa rjts a ccounted for almost 1 5% of all Part D partici- 
pants during th*e third year. * DCiring that same year, high school work 
experience participants were reported to' rep resent- 8. 4% of cdl high school 
participap|s, or 1. Z% of total participants. * , 

Projects reporting work experience activity costs reported that on the 
avetage 16. 9% of the projects* Part D expenditures were devoted to these 
activities the first year, 17. 5% was reported for the second year, and 
19. 5% the final year. (See Table 61, below, ) The projects reporting work 
experience participants reported an average of 84 per project for the^first 
year, lb4 the second year, andcl28 the third year. 



TABLE 61 

CO\tPARISON' OF WORK EXPERIENCE ACTTVITTES VERSUS COSTS 





Program 
Year I 


Program 
Year II 


Program 
Year III 






Average Percent of Part D Project Cost 


• 16. 9'-^ 


17. 5--S 


19 5?< 






dvCTz^c Number of Partiripants per Project 


84 


110 


128 




t 




h-" 











From Table 57, it can be seen that considerable variation exists in the 
percent ^of Part D funds devoted to work expe rience activities. Frequently, 
this variation is due to the liature of the activities Included in the work » 
experience component. Some projects were able to expand pre-existing 
work experience activities to include project participants or tu meet project 
needs. Others attempted to utilize personnel on a limited basis to provide 
work experience opportunity for a limited number of participants. These 
projects typically did not report w|i|t experience costs exceeding 10% of the 
total project costs. Some projects employed work experie^jce coordinators 
^ or job placement specialists on a full-time basis to implement the project 
work experience componeht. Where this occurred, the projects reported 
work experience activity costs that ranged from 15 to 25% of bne total 
project co3ts. , > 

C3ne uroject included intensive skill training and counseling as an 
integral part of the work experience component. All participants were 

provided with all treatments and no other students were involved in the 

project even in aJLimited way. A a would be expected, thi^ intensive 

approach was also relatively e:6pensive, and this project repprted the' 

highest percent cost of work experience activities of the projects studied 

(89.3%). . 13-. 

With^only two exceptions, projects reporting costs for work experience 
^-aciivity did so for aA three years; all of them reported such co.^ts for the 
final two years. The three year trend shown in Table 61 indicates that 
projects devoted a ^lightly larger percent of their total budget to work ^ 
expcilcnce activities and tended to involve nAore students in these activities 
each year. This reflects the overall trend for all high school activities. 
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b) High School Job Entry Skill Training 

A total of jZZ project^ reported Part D job entry skill training costs as 
part oJ'iHe total Part D project costs. These projects reported that 
skill' training activities accounted for an average of 21% of the first year 
/' Part D project costs; 17. 6% of second year costs, and 13. 6% of third 
yea-r costs. Table 52 compares the average percenta^ge of costs by year 
to the average number of skill training participants for the same period'. 





TA>5LE 62 

CONIPARISON OF SKILL TRAINIKG AtTTVITIES VERSUS COSTS 










Program 
Ycarl 


Program 
Year II 


Program 
Year III 


r 




Average Percent of Tot^i! Part D costs 


21. 0-^ 


17. 6S 


13.6- 






Average Number ol Participants per Project 


338 


408 


355 . 





c) 



During the third year, skill training participants coniprised 19.8% of*all 
high school participants in ^the projects studied and 2. 97o of ail participants. 
As can be se^n from the table, projects repdrbj^d that skill training activities 
ac,counted for progressively smaller percentages of the total project costs 
v,hile the average number oi skill training p,articipants remained much the 
sarAe bebx^een the first year and the third, with a slight increase during 
the 'Second year. 

As indicated by the reported activity costs, skill training activities, where 
they were undertaken, tended to be three year efforts. Only two projects 
reporting costs indicated these costs for fewer than three years; one for ' 
the final two project years, knd the other for only the last project year. 



Senior High/School Familiarization Through Infusion by Participating 
Teachers ' / I 



Over one -half of the third year high school participants (53. ^^o) were 
involved in occupational familiarization -or orientation activities pro'^id.ed 
^ by participating, teachers. This represents 7.8% of all reported third 
year participants. 



Thirty-one projects reported hi^ school participants in this treatment 
area. Of these, 29 provided activity cost estimates. 'From the informa- 
tion provided it was found that these activities constitiitfed an average of 
16^ 6% of the total first year Part D costs. For the second year the figure 
w^s found to be 15. 3% and for the third year it was 20. 5% of the total. 
From Ttxhle below,, it can be se^n that the average number of students^ 
remained stable^ oVer the three project years. 



TABLE 63 

COMPARISON OF FAMIUARI2ATIO^^ ACTIVITIES VERSUS COSTS 



^ '' ' 


ft^ogram 
t Year I 


Program 
Year H 


Program 
Year lU . 


Average Percent of Total Part D Costs 
Aycrcge Number oLPaKldpants per Project 


16. 6^ 
675 


15.3^ 
685 


20. 5?^ 
673* ^ 



- \11> - 



The three-year trend indicates essentially no change in the average number 
• of pariiciijants fyftm year to year. During that period, projects indicated 
an overall increase in the average percent of total expenditures devoted to 
high school familiarization activities.with the second^ear being slightly 
. below the figure reported for the first year. * ' 

It was observed earlier that with four exceptions the work experience 
and skill training activity costs tended to be reported for Jhe full three * ' 
year perio'd of Pa-rt D support. High school familiarization activities 
provided by teachers, however, tended to follow the more variable pattern 
evidenced at the elementary and junior high levels. Five of the projects re- 
porting elementary activity costs in this area reported no costs in the first 
* year. Two of these indicated third year costs only. This would suggest / 
that projects required more in^the way of start-up tinrve far familia^ation 
activities at all levels than for work experience and skill trairttn^actS+ties. 
This is at least in part explained by the newness of the familiarization 
activities; projects reported ^hat to ipvpri:ement these activities required 
devoting considerable time and resources to planning, material and 
curricula development, and stafft^raining. Work experience programs 
ana job entry skill training programs, w^ic^x^y^pe frequently included in 
the traditional curricula, didr^ot reqirrfe'^'much time for these prepara- 
tory activities. Thus, projects were able to expaiid or "buy into" 
ongoing skill training or work experience effqrts, .and required little by 
way of start-up and development costs. 

^) High School Occupational Counseling and Guidance Activities 

Eighteen projects* r^poiA^'d third year expenditures for high school level 
occupational counseling 'and guidance activities. Participants in these 
'activities represented 14. 37o of the total third year high school participants 
- and 2. 1% of all the third year Part D participants. As shown in Table 64, 
projects reported that both c'osts and participants for counseling, and 
guidance activities gradually increased each year with the greatest 
^increase occurring between years one and two. 



















TABLE 64 ^iMfe 
CONIPARISO.V OF COUNSaiXG AXD CUIDANCE ACnvIT^S VEEISUS COSTS 


k 


> 






Program 
Year I 


j EVogram 
■ Year 11 


Program 
Year HI 








Average Percent of Total Part D Costs 






14,354 




9 


Average Number of Participants per Project ' 


427 


1 597 


625 







Some grantees were found to l^ve employed new personnel to initiate occuna- 
tional guidance or placement activities. Some projects emphasized 
occupational guidance at the high school level as opgosed to infusion 
activities. These^projects tenSed to.report higher proportional costs fr/r 
tliese activities than other projects. Generally^ these projects. are the 
oifes reportii^g, counseling and guidance, costs in excess of 15%' of the total 
project costs 
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Several projects attempted to involve the regular counseling staff in 
occupational guidance or in gxoup career education activities. This - 
approach was usually aimed at involving greater numbers of guidance 
personnel than the approaches described above. Projects taking this 
approach tended to report lower costs, i. e. , 5 to 10% of the total 
budget. 

e) Other Project AjCtivities ^ *" ■ 

Nineteen projects reporteS%ignificant project activities th^ could not 
be described as belonging to any- of the above activity areas. Some 3. 9% 
of the total high school level participants were involved ih these activities. 
This represents 0. 6% of the total third year Part D participants. 

• - ^ f * " 

Of the 19 projects, there were five which reported activities directly 
treating students. " These activities cost an average of 88. 6% of the 
total project costs the first project* year. For these projects the 
second year- costs av-eraged 81. 8% and avefaged 79. 0% the final year. 

* In each of tlie projects the activitfes related to the creation of an 
alte^ativ^e high s-chool setting for dropouts, potential' dropouts, or. 
individuals whose educational needs were not being met by the traditional 

' educational delivery system. . . 



\ 



From Table 65 it can be seen that the numbeif of participants per year for 
these activities was low in four of the five projeWs, ^nd that their cost wag 
a mtich greater proportion of total project costs than for Che high school 
activities reported earlier. The relatively high costs are explained by the 
unique nature of these activities. They were innovations that could - 

'not easily tie into ongoing activities, and they were carried on^in a 
setting outside that of the traditional' educational delivery system. Thus, 
they were not amenable to the use 'pf regularly employed grajitee staif 
on a part-time basis. They were also cornprehensive in nature, / 
attempting to meet the total educational nee<is of the participants ini/olved. 
For these reasons, they were Relatively expensive when compared to 

* other Part D Jrtivities.» 



TABLE 65 

"OTHER" PROJECT ACTIVITIES ' 
Comparison of Participants to' Percent Cost of Activity, by Project 






Profects 






I 


II - 


ni 


IV 


V 


Project Year I: > 












Nuinbev of Participants 


0 


700 ' 


0 




156 


Percent of Part D Costs 


89,7^. 


80.9%. 


100% 


72.4% 


> 100%. 


Project Year Ih 


r 








f V 


Numbefy of Participants 


70 


600 


36 . 


232 * 


201 


/ " Percent bf Part D Costs 


^ 79. 5% 


80.9%' 


100% 


"65.7% 


82.8% 


Project Year 111: \ 












Number of Vartici pants ' 


90 


600 


60 


180 


180 


Percent of Part D Costs 




80.9% 


100% 


78.6% 


74.8% 










-r 
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No general pattern of increase or decrease in the nximber of participants 
is^evident for these activities. Also, no such patterns emerged with, 
respect to percent cost of these activities. Two projects reported no 
change in pe*^^^t of total Part D costs during the three year period, 
while another indicated decreases. oneM;he decrease represented 
use of another grant to add a K through 12 program to the original 
program; the other project sjj»f^y devoted more resources to its elemen- 
tary school activities. \. ^ * 

Three of the rtineteen projects with ''other" activities reported efforts 
which could hot be related to specific students; while these costs were 
greatly below the five discussed a^ove, they represented a sizeable ' 
proportion of total project expenditures. The cost of these activijties 
was reported to be in the range of 30 to 35% of the total Part D cost for 
each project. One of the projects reported the costs of cjeveloping a 
career education TV film and holding a state career education conference'. 
Another project reported a summer orientation and field' trips to industry 
program, as costs in this area. The third project reported the develop- 
ment of ap occufjfational information media center in this category. One 
of these'projects identified some 350 participants but only for one summer. 
One indicated a^'potential" number of participants and the remaining 
project; t&uld not estimate impact of these activities, 

* ' ' * \ . - 

Tile remamiiig 11 projects reported costs in the "other" activity cost 
category that did not reflect any student related activities. Us^ually 
these costs include^d administrative costs, Evaluation costs, or teacher 
training costs. The range of these costs ran from a high of 100% pf the 
project costs to i% of the total. 



4. "Summary of Cost of Project Activities Findings 

^\ 

Table 66 compares the third year activity l?Skts with the third- year participants, 
'^As above, the costs are expres-sed as the average perceYit of total project costs 
devoted to the given activity for projects reporting that activity. Participants 
are expressed as a percent of the total number of third year Part D project 
' participants, ' ' *^ 



TABLE 66 

THIRD YEAR' COST*AND NUMBER OF PARTICIPANTS AS PERCENTS O'F PROCRAM TOtALS 



Average Percent of Total Part D Cost 
Percent of Total Participants ' 



Activities 



Elementary 



24. 5% 
5^. 1% 



Junior High , 



27.6^0 
33 ^% 



All Senior H: 



14.4% 



ERiC 







^Vior Hi eh 


Subcateeories 








Work ' 
' Experience 


Skill 
Training; 


Participating 
Teacher 


Participating 
Counselor 


Other 


Numoer of^^f^jects tleporting 

<a - . 

Average Percent of Total Part D Cost 
Percent of Total Participants 


» 26 

19.5^ 
1.2%' 


• 23 

13.6% 
2.9% 


31 

' 20.5$^ 
7.89^ 


22 

14, 2% 
: 2,1% 


8 

71.3% 
0.6% 


Because not all projects, had all sjcuvitici, it ii not appropriate to add across columns. ' 
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It can be seen that elementary school participants comprised 52.2% of the total 
* third-year Part D participants and that projects reported that elementary school 
activities cost an average of 24, 5% of the total project costs for that year. 
Junior high school activities required an average of 27. 6% of project Part D 
resources to treat 53. 5% of the participants; 47. 9% of the Part D project cosfts 
were devoted to high school level activities that in total affected pnly 14.4 % of 
the third-year participants. 

p 

Dur^g the three-year period of Part D funding, projects clearly reserved the 
largest share of their resources for high school activities and the smallest for 
elementary activities. The three-year trend was to decrease the proportion 
of resou^rces devoted to elementary and junior high activities and to increase 
the proportion for high school activities. , 

The decrease was sharpest for elementary activities during the second year 
with tbe expenditures only slightly lower the third year. At the junior high 
level there was a second year increase with a decrease the final year. From 
this it woulcf appear that projects tended to emphasize elementary activities 
the first year and junior high activities the second. Expenditures for senior^ 
hi^h school activities were expanded as the others were decreased 
# * 

At the high school level projects reporting "other activities," 
tended to devote most of their Part D resources to thos(> actlA/'ities. ' High 
school familiarization activities designed to infuse career education concepts 
into the regular curricula and work expe rience activities rs&iked next, and 
were reported to cost projects approximately 20% of their third year expen- ' 
ditures. Counseling* and guidance activities and job entry skill training 
activities followed at a reported cost to proj'ects' with tho§e components of 
14. 2% and 13.6%, respectively. ^ n " 

During the bourse of the three year effort the resources expended for skill 
training activities decreased significantly^ Skill training activities supported 
with project Part D fundawere the only high school activities that reflected^a 
decrease during both-tfJe ^econd and third years^' The three year rate of 
decrease was oyer twice that of elem^entary activities. This was the highest 
rate of overall decrease indicated for all activity areas^. 

Interestingly, infusion activities by participating teachers at the high school 
level were reported to have a lower percentage of support the second project 
year than during the first year. A substantial increase in percentage was 
reported for the third year. This change, from 15. 3% to 20.5%, wa3f>the , 
largest single year change reported for any activities. 

\ ' 

Overall, projects repoj^'ted an increase in the average number of participants 
for each ac,tivity during the three-year term of Part D support with the exception 
of high schqol infusion activities. In this*area, the number of participants was 
virtually the same in the third year as in the first year. The, number of third 
year participants was reported to be below the second year figure by an'a^era^e 
of 12 participants per project. .This means that during the third year this 
activity, was reported .to cost project over 1/3 more, proportionately, while the 
numbe^B of participants involved actually decreased^slightly. There was no^ 
other activity area showing an overall decrease in the number of students par-, 
tic ipa ting,, a^nd only one other for which a one-yfear reduction wag indicated. 
This was job entry skill tr^Jiiing, where a r^ducltion in percentage of funds was 
reported for the third year, from 17. 6% to 13. 6%, coupled with a 13%.r eduction 
in the average number' 6f participants. 
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Elementary school participants were repor^pH to have been increased by an 
average 47,% per project during the three years of Part.D support. For the 
same period projects reported a decrease in the'proportion of Part D resources 
devoted to these activities* f^om 28% 16 24. 5%. The;average number of junior 
hish school participants increased by nearly 33^c while the proportion of the 
Part D expenditures directed at these activities was reduced slightly (from 
29. 3^0 to 27. 6%). At the senior high school level an average 31% increase in 
participants was reported coupled with an increase in the proportion of funds^ * 
devoted to these activities^ from 42. 1% to 47.9%. 

r * 

A review of the activities at the high school level reveals that work experience 
activities had the highest rate of increase in average number of participants 
for the three years. With' the lowest average number, of participants per project 
reported during the first year, there was a 52^0 increase by the third year. The 
average costs went from 16.9% to 19.5%. Counseling and guidance activities 
reflected a 46% increase in the average number of participants during the three 
years, coupled with an increase in percent of costs from 11.5% to 14.3%. Skill 
training aqfivities, as indicated ^arlier , were reported to have experienced an 
:icrease in participants. 




The figure below rank orders activities by year from highest to lowest, accord- 
ing to the averz^ge relative cost of activity. This figure graphically illustrates 
the third year shift in activity expendiJJure emphasis at the high school level. 



RANK ORDER OF/VERAGE HIGH SCHOOL ACIIVITY COST, BY YEAR 



Rank 

1 (hi^h) 

2 

3 

4 (low) 



Program Year I 

skill Training 
Work Expefienee 
Participating Teacher 
Counseling G Guidance 



ft-ogram Year 11 

Skill Training 
Work Experience 
'Participating Teacher 
Counseling GvGuidancc 



Program Year III 

Participating Teacher 
Work Experience 
Counsebis and Guidance 
Skill Training 



It is apparent Chat the third year s'aw a shift in emphasis toward class ro<sfm 
familiarization and occupational guidance and counseling and away from skill 
training. One possible explanation for this relates to the general availability 
*'of skill training !fesources in most areas. Projects may have elected to rely 
on traditional training resources, faking them available to students not taking a 
full vocational course of^ training, the reby 'f reeirfg resource's for othet activities. 

Another facto,r that may -be involved is the relative cost*c?f skill training and 
work experience activities when compared to the others. Both are decidedly 
more expensive whea a cost per pupil ratio is applied. 'Classroom infusion 
techniques and the guidance approach to caree.r educatipn^have tjie p'otential 
of touching, many more students at a lower cdst ratio. That work experience 
activities did not show a similar reduction would seem £o be a-function of 'the 
limited resources for these activities available from 'other sources in fhany 
of the grantees. Below we present an analysis of these cost findings by .relat- 
' ingthem to the results (5btained f9r outcome ques^tions. 
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* C. Results and Conclusions on Cost/Outcome Findings ' 

In Chapter IX it was shown that some projects were more effective in producing . 
student outcomes for a given activity than were others. W^en these projects' 
average number of participants per year for'^these activit/ areas, was compared to 
the overall average per project for the same activities, if w,as found that the* 
projects differed from the average in numlfer of participants per project for a , 
given activity an^/or,in the rate of change in the, number of participants in the 
^activity from one year to the next. A similar an^lysi^ of these projects with 
respect to cost was also carried out. *The results of this analysis are reported 
below |or each activity area. • • ^ 



1. Elementary Activities 



The .ten projects with the most favorable respon&es with r^sp^cl/to student ou\:- 
comes at the elementary level were found to have a lower average number of 
elementary participant? per*year than was reported across all projects. While 
averaging fewofr elementary participants per year, these projects reported a 
higher proportion of expenditures devoted to elementary activities than was 
reported for all projects, as indicated in Table 67, below. 



TABLE 67 

Comparison of Average Pefcenfof Total Expenditures f"br Elementary 
Activities Per I^roject to the Averagre for the }0 




^ — « 

Reportinfir Category Average % of Total Expenditures 


10 Most Favorable I^rojects 


Yeair I 


Year II 


Year in 


39,1 ' - 


31,8 


3-0.3 


Ove ra^l^Hrage Per Project 


28.8 ' 


24.8 


24.5 



Juni|j)r High School Activities 

~y. ~ , ^ ' 

As indicated in "^able 68, the ten projects with the most favorable responses to 
queistions at thegunior Kigh school level had an average expenditure for thos^ 
activities that fluctuated with respect tc5 the average overall project expenditures 
foiAthese activities. During the first year, junior high -activities cost these* 10 
projects a smaller percent of their tot;al Part D expenditures than was experi- 

-enced overall. During the second year, these 10 projects increased the p|*opor- 
tion of these expei^tures more than the overall average, to^e/point that their 

, average percentage\>f costs was higher. Iq^the third year, however, their 
average percent of expenditures was virt\|(ajj 
all projects. 



the same as the average for 



1 w 



-7— ^ TABLE 68 ' 

Comparison of Average Percent of Total Expenditures for Junior High 
School Activities Per Proje6t to the Average for the 10 
Most Favorable P-rojects 



Roportinor Catecrory 


* f 

Average % of Total E«penditures 




Year I 


Year IL 


Ye^r III 


10 Most Favorable Projects , 


21.4 


38.3 


27.3 , 


Overall Average Per Project 


29^. 3 


* 3K0 


27.6 











Tht?se cost data become significant when cornered to the ave rage ,nurnbe r of 
junior high participants per project. The 10 most favorably responding projects, 
as pointed out in Chapter IX, averaged at least * 50% fewer participants 
per project per year than was reported for all projects. . 

3. Senior High School Activities 

.Findings with respect to the senior high school activity categor^ies are presen- 
ted below, ^' ] , .- 

a) Woi;k Experience Activities • 

In Chapter DC it was rep'orted that the number of paijticipants in work 
experience activities increased for both those projects with the 'mosj; 
favcrrable responses to student outcome questions and for all projects 
for all three p«roject years. As indicated in Table 69, the seven'most 
favorable projects with respect to student outcomes tended to spend a 
' smaller portion of their Part D monies for work experience activities 

than was reported for all projects. This ]relationship continued through- 
out the t^re e-year p^rpject period with both the mo'st favorable group of 
projects and all projects taken together gradually increasing the per- 
centage of expenditures arid the number of f)articipants each year. 

♦ 



TAbLe 69 

Comparison of Average FJercent of Total Exp(hi^tures for 12t± Grade 
~ rience Activities Per Project to the Average lor 



Work Experience Activities i^er Projc 

the 7 Most Favorable Projqfcts 



Reporting Category 


Average % of Tot^l Expenditures 




Year I 


Year II 


Year III 


Seven Most Favorable 'Projects 


12.1 


13. 0 


14.9 


d^erailAvera^ge Per^Prdject 


16.9 


17.5 


' 19.5 ^ ^ 






• * 
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In the pre^eeding chapter it was pointed out that there were a totdl of six 
projects \yhich skillet raining activities resulted in the production of 
favorabl^^tudent outcomes. Those projects, ^it was noted, differed from 
the average of all projects in that they reported a higher number of partici- 
pants^^per project; i, e; , the projects with favorable outcomes increased 
skill/training participants by 30% over the three years as compared to an 
overall increase of only 5% for the three-year period. 

Table 70 shows, in terms of cost, the thjree year trend with regard to> 
expenditures for skill training activities was a progressive proportionate 
decrease for ail projects. The expenditure data suggest similar experi- 
ences for Both groups in terms of trends and actual expenditure levels. 
The major difference betweeathe two groups is that the most favprable 
projects involved twice as many participants in skill training activities for 
approximately the same relative cost as was the case for all projects. 



> TABLE 70. 

Comparison of Average Percent of Total Expendj.tures for 12th Grade 
<=5 ' Skill Training Activities Per Project to the^^erag^ for the 
Six J^4ost Favoi'able. Projects 





Reporting Category 


Average % of Total Expenditures 






Year I 


Year II 


Year III ' 




^iX'Most Favorable Proiects 


20 


15.6 


14.9* 


> 


1 * 

Pve rail A^ye rage Pei* Project '^^ 


*'.21.0 


17.6 


13. 6_ 



* c) High School Faiyfiliarization Activities through Participating Teachers 

Five {projects were foxmd to have "be en most effective in producing student 
outcomes as a result of the high school familiarization activities of partici- 
pating teachers. These projects reflected an overall thre'^-year increase 
in the average humber of participants per prpject of^l53%, while the ' 
^ general experience" of Part D projects was to show no increase in the 
number of participants during the three years. 

^ . . I • 

Table 71 shows that the three-year expenditure pattern for these activities 
was fotmd to be the reverse of the participa^tion trenci Keported* earlier. It 
was fotmd th§t on the average, all projects increased their percentage^ of 
expenditures for the§e activities during the^ three-year term of Part D ' 
support. * The five with the' most fsj^vorable responses to student outcome ^ 
quesfcionsT,* however, 'dec f-e*5tded their average p^e^e'nta^e of expenditure • ' 
^, for each gj^the-three yeaVs^. Indeed, in the-^mrd year these five projects 
reduced their average percent of c^ts^elow the-^verall ayerege: f 



J 



Tlie overall ^perience^^at^^nie high acho^^vel differs frJswn that at the 
elemen^rj^^Sd jujii<3f¥^igh' school levels with respect to fatniliarization 
activitie^^'^^^ifever, the projects ^foirnd to be. most effectiyeNj^i producing 
studen^^ofitcoraes were found to report three-yelar cost and paxJleipant 
rends which parallel the experience at the elementary and jimior 
levels^. ' , • ' 
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TABLE 71 



7 



Comparison ^d£ Average Pertent of Total Expenditures for 12th Grade.: 
Familiaili^atiion Activities Per Project to the Average for the 



Five Mist Favorable Projects 



Reporting Catecory \ * . • 


Averap:e 


% of Total Expenditures 




Year I 


Year H 


Year HI 


\j- '/■■ 

Fiye Most Favorable Projects / 


^ 22. 1 


20.9 


18.3 


/ T — 

Ove rail Average Per P!roject , / 


16.6 


^ 15.3 


20. 5 



D. Summar\/and Conclusions 



Projects which were most effective ini)roducing student outcomes as a result of the 
activities of participating elementary And secondary teachers and counselors, i.e.; 
familiarization, were found to differ/from all projects at all levels (elenlentary , 
junior high, and senior high). At thre elementary level the projects with the greatest 
number of favorable responses in^ilved fewei^ participants per project than did all 
projects. These projects also devoted a larger proportion of their Part D funds to 
elementary activities than did al/projects, - ' 

' . . It-' 

At the junior high level, the projects most effective in producing student outcomes 
increased their proportionate,expenditures during the three-year Rtrt D support. 
On the other hand, t^e trend for all p rojectS^reflected an overall decrease in the 
expenditure percentage for these activities. The projects with the most favorable / 
'responses most noticeably.differed from ^11 projects in the number of participant? / 
per project. For each of the three years Jhese 10 projects involved 50% or fewer 
participants than did all projects . \ 

At the senior high sche>ol level, those projects f ound>6 be most effective in procibc-. 
ing student outcomes through the familiarization activities of participaj:in g teachers 
showed a pattern of <iecreased cost percentage 'durir-g the three-year term of Part D 
.^upport. This wasycoupled witV- an, overall^ ^creas^ in the number of partic i^^ants ' 
for these activitie/. This compares with an overall increase in the expenciitur e 
percentage level ^or all projects, coupled with no gain in the number of participants 
per project. 

The difference's between the projects f oun<i to be most effective in prpdacirig student 
outcomes as jL result of skill training activities and of work experien'ce activities 
•and all projf/cts with those activities was not as proJiounced as wer6 ^fc^ie differences 
cited for fajhailiar ization activities. The projects with the greateat number of favor- 
able respo4ises °in the work experience activity area reflected both .4 slightly lower 
percentage of expenditure^ afid f«wer participants than did all projects with work 
experience activities. ✓ ' ■ ' /• 'f^* 

The proportion of expendit;ur es and the three-yea^ trend of those jDrop^-rtions did not 
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vary significantly between the most favorable projects in skiU training activities and 
all iSrojects with those activities. The mos't favorable projects, did, however, 
involve considerably more students per project than did tfijs' average for all projects. 

^' / 

}iven. the initia*lly higher percentage of costs and,th^ lower level of participant involve- 
ment in the projects with the Jnost favorable responsesyas compared to all projects, it 
mdst be concluded that the most favorable projects invested proportionately more ti^e 
and resources in start-up activities than did other projects. It appears, therefore, 
that a major factor in the success or lack of success of a project in producing student 
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outcomes as a result of familiarization activities 4s tlxe propD^tion of investment and 
activity prior to. full-scale project implementation; ' * . • 

In general, work experience activities and job entry skil]. training, activities were 
less effective in producing student outcomes than famili^jizsttion activities. It is 
possible, however^, t^t the overall shift of resources toward high school familiar- 
ization activities noted in the third jfroj^ect year contribute^lo a lower'level of 
student outcpme resists for the work experience .and skill tijaining student group. 
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, - CHAPTER XI: STATE ADMINISTERED PART D(d)^FUNDS 



A? Background. and Purpose 



Pcirt D of ^ho Vocational Education Act, as amended in r968, was designed to stimu 
late th(? development pf new appr oachesv to linking school and the world of work, 
thereby assisting young people to identify ^nd prepare for careers most suited to 
their .personal aspir^i^ons^d talents. Con^res s'directed that funds appropriated 
iTtvthitS section of the Actbe expended to redyce youth unemployment through 
exemplary efforts which accomplish one or more of the following: 

/ 

• familiarize elementarv and secondary school students with the broad range 
of occupations for whifth special skills are required and the requisites for 
carec^rs in such occupations;^ 



• provide students with educational experiences through work during the 
school year or in the summer; 

• prpvide intensive occupational guidance and counseling during the last' years 
yf school and for initial jobs placement; 

/ \ 

• .broaden or improve vocational education curriculua; 
/ ' 

exchange personnel between 3 chools and other agencies, institutions, ♦or 
organization^ participating in activities to achieve the purposes of the Act, 
including manpower agencies and industry; ^ - . - 




y increase educational attainment of young workers released from their jobs; 

/ • motivate and provide pre-professional preparation for potential teachers 
for vocational education; and y 

• provide otiier activities which ar^ consistent with the {>urpose of the Act. 

Congress stipulated t?hat half of the appropriated funds were to be expended by the 
Department of Health, Education and Welfare in each state and territory and that 
•the othe^half were* to be allocated to the states and territories directly. 
The^ portion of the Part D appropriation dispersed by the states and territories was 
not constrained by the guidelines applicable to the projects funded directly by 
^USOE.* The states wire obliged simply tn utilize the •funds^.in accordance with the 
purposes and pruvisidns of the legislatioi and the administrative and reportkig 
pfrovisiQns specified in the Federal Regime r. ''^k As a result, both the nature and 
duration of the state administered projects, ^Part D Sec 142(d), in many states 
varied markedly frofn the projects funded under the federally controlled portion 
of the Act. They also varied considerably from-^^tate to state. 
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W^hile the major emphasis of this evaluation effort was on the federally supported 
projects, there was an interest in determining how the ^state -administered portion 
of the Part t> funds was utilized* More specifically, USOE wa« interested in deter- 
mining the objectives of the state funded projects, -and tho*strategies used to 
achieve these objectives. Given the level of detaiLdesired and tSe fact that c^er 
1,000 grants had been awarded by the states during the first three years of the Part 
D effort, it was determined by USOE that the source of information on state admin- 
istered projects would be state level personnel only. The purpose of this 
chapter is to present and discuss the information collected on the state funded 
projects. In acjdition, a summary of relevant activities of tfie Research Coordinat- 
ing Units in the states visited is presented.* , - * ' 

B. Study Methodology^ ♦ v 

, To secure the information necessary to answer the questiAis posed by the study 
^regarding the state administered Part D funds, the first step was to revif^w the 
information already gathered by USOE. One requirement imposed, on the states 
by the Office of Education was that a copy of each state -approved proposal be 
submitted to the Division of Vocational and Technical Education of USOE (DVTE) 
within 15 days of the award of a grant or contract. Based on the information so 
provided, QVTE published a booklet, "State Administered Exemplary Projects in 
^ .Vocational Education, " in the Spring of 1973. This publication contained, for each 
state, a list of the project titles along with the name of the organization receiving 

^he grant and the amount and duration of the g^nt. Review and analysis of the 
information in this publication was carried out during August and September 1973. 

*' > 

The review of the information available within USOE made it clear that additional 
data from state departments of education were necessary to respond to the study 
questions. Specifically, the following was necessary: 

• verification of the information contained in the TJSOE publication; 

• identification of the objectives of eac^project; 

^ •an indication .qf the criteria used in awarding Part D grants; 

• an^indication of the ove rail funding strategy; 

an indication, of the relationship between the state departments, of education 
and the recipients of the Part D grants; and 

^ an indicatibn^ of which projects and acti\jities were judged to bje worthy of 
replication, " , . ^ 

It was assumed frbip the start that collection, of^ata pertaining t^tate projects 
would be collectefa in the course of gathering data pertaining to tfte federally 
a,dministe red Part D projects. Consequently, it was concluded that data would 
bfe obtained from the fo^ty-nine states, plus the* District of Columbia, where first 
r^mid Part D projects were to be studied; since a study team would not visit 
Hawaii, data su()piementing that already >n. USOE files w.culd not be obtaine,d in 
that state. In add,ition, it^tVas concluded that the information.wpvdd be obtained 
by the meniber of the study team who was scheduled to interview TSate personnel 
in the course of 'obtaining data on the federally supported projects and tha£ the 
primary mode Of data collection would be an interview with the in^ividual(s) 
responsible for the state administere(^ Part D effort. For the mo si part, the individ 



ual within the state department most conversant with the federally supported 
projeci^wa^ al^o the individual responsible for the.^state 'funding p'rocess. The 
interviews wejre conducted in the Spri ng of 1974. * . 



To facilitate the >cter view process a single interview schedule wait prepared to 
obtain data on both>he federal and state projects. In addition to the interview 
schedule, two forms rer each state were prepared, These^jCprms were sent to the 
state departments for completion prior to the arrival of the study team intervieweT. 
On tho first form the state department offifial(s) most fanniliar with the state- 
ftinc}<?d projects were asked to identify thc^ jfeUf^ctives of each grant, lii essent^O, . . 
the form was a matrix with each of the statt^- funded projects listed as the ro\y>s anp 
eight objectives, taken from the federal register, forming the columns. The state 
representative was asked to indicate which objective^ applied to each project and 
to up-date and verify the list of projects on the form. On-*he second form the 
state representative was asked to indicate the anrfbunt of the grant awarded by fiscal 
year, and the number of participants served by thQ project. 

in, approximately 50% of the cases these two forms wcne completed at leas^ m part 
by ^he.time the interview took place. In these cases the interview ir\pl uded a 
review of the information contained on the forms to verify that the ir.ct ructions were 
clear and the inforvmation was recorded consistently. In some other cases th^ 
forms v^e re completed by the interviewee or an* assistant during the course of the 
interview. In the remaining cases the interviewee indicated that more time was 
necessary to complete the forms and that they/ would be returned to the study team 
at a later date. In one state the information was not received despite several 
follow-up requests. * * « 

In general, the represent';^ives of the stat^ departments were both knowledgeable 
and cooperative. For. the most part they wete quite familiar*with the Part D 
effort and willing to assist the study. In some cases, however, there had been 

substantial staff turnover within th« st^te department since the time the Part D , 

effort began and the interviewees were understandabi^' hesitant in responding to \ 
questions regarding events and criteria which preceded their involvement in the 
effort. A second qualification which must be placed on the data obtained from the 
interviewees is that in most cases the st*ate-supported grants were awarded by a 
committee and ,tSe Criteria were not formally recorded. As a result, the state- 
ments reflecting the funding criteria used reflect the judgment^ of only one of the 
several individuals involved in the project selection p^rocess. 

C. Findings. 

1; Allocation and Obligation of Funds 

The available data concerning each «;tatc*s total allocation and obligation of 
' funds, Iv/i FY /u, FY 71, and fV 72 ax;e^shown in Table 72, together with three- 
^ year totals, the percentage of each year's allocation that was obligated, and' 
the percentage of the total sallocation that was obligated. As shown in Table 73, 
one-year allocations ranged^£roi|^ a law of $102,000 (Alaska, FY 70) to a high 
of S348, 000 (California, FY 72), with^median allocations %f S117, 000 in FY 70 
arid S139, 000 in FY 71 and FY 72. Obligations ranged from a low of $44, 000 
(Nevada, FY 70) to a. high of $421, 000 (Floti^da, t'Y 72), with n^edians of ' 
$115,000, $144,000, and $255,000 in FY 70, FY 71, and FY 72 respectively. 
Both the rninimum and maximum obligations .-wC re lowest in FY '70, as we r-Q 
the minirr\um anH maximum allocations^^ *Xhe lowest reported percentage 
of allocated funds thilt were obligated in one year was 3 5% (Illinois, « FY TO), 
while t\iC highest r^portdd percentage was 238To (-Florida, FY 72). On a mree- 
year total basis, the median pe rcentage obligated was 95.6%. There way no 
apparent relation between the amouht^llocated and the percentage obligated; 
for example, the state with the lowes t Ihree-year allocation, Alaska, -<^Ugated 
100% of its funds and tKe state with the highest three-year allocation, ^ 
California, ^obligated 99. 5?W>^ its funds, i 
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TABLE 73 

RANGE AN'D. M E D F .\ N OF STATE ALLOCATIONS 


AND OBLIGATIONS* 




FY-'O 


FY-71 




■J2 : 


3-Year Total 


Low 

Mediin 


^ $102,000 
• ' 2U.OO0 
U'. 000 


AUOCATION' 
^ $104,000 
346. ^Ool 
139.000 




$104,^)0 
348,000 
139.000 


$310,000 ' 
905,000 
389,000 


lev/ 


S 44.000 
'234.000 

ri6,ooo 


' " ' ~ c 

OBLIGATION 
$ 91 . 000^ 
364,000 , 
144,000 


/ 

S 79/000 
42l/,00Q . 
2S5,OOQr 


$246,000 
919,000 
412,000 


Median 


' 35.\ 
^2 i. 1 
93.0 


PERCENTAGE OBLK 
64. 3 
1 38. 1 
95.1 


:ated 


59.9 
237.7 

9S.7 


64.9 
1 56, 6 , 
95.6 


■ All dollar figures are to the nearest Sl ,000 









Approximately one-fourth of the states (14) obligated less thanv90% of their ^ 
three -year allocation, and eight states obligated over 100% of their allocation' 
liy contributing funds from other sources, at leajS-t temporarily. ; 




\V;th respect to' those states reporting pbligations pf avrr1*W7rx?f the amount 
allocated for a year, several explanations are'possible. First it should be^ . 
, j:ioted that the federal legislation permits expenditure of funds, allocate^ for 
one fiscal year m the next fiscal year;''"'3jr-tK^'^asje of Coldia^lo, for example, 
this appeats to be the case. Colorada repgxjj^lobjtigatwi of its alloca- 

tion in FY 70, 17^7% in FY 71, and^lZZ^O^^h^^- 72, It appears* in this case 
that what was unobligated in FY 70 was applied to the total amount available 
for FY 71, and the unobligated balance of the available funds for FY 71,iAand ' 
the unobligated balance of the available funds for FY 71 (new federal funds 
plus unobligated balance from FY 71) was a'ppLied to FY 7Z^^ Assuming\this 
approach, it ,is quite possible for Colorado^ obiigate'Jj22% of the FY 7^ f 
federal allocation (new money) a^d still show after three years, e total obliga- 
tion of 92. 4% of the total federal allocation, ilarrying over funds may»lso 
partially explain why half the states obligated 95;6%or less of their three-year 
allocation, * 



A second possible pxplctndtion is illustrated by Minnesota, In Minnesota, 
99, 4% of the allocation was report^ed as ob>ligated in FY-70, 109. 5% in FY-71, 
and 59, 9% in FY-72. In spite of the obligation in FY-71 of 109, 5% of the 
ajjocated funds .when virtually no funds from FY-70 remained unobligated, 
the three y^^'* tc^tal obilgeicion was less than th^ total federal allocation. In 
this case, it appears that funds, f]^m some other source were used' to supple- 
ment the available Part D allocation with the expectation that in the lon^'nin 
Parf D would cover the obligation. This may be the explanation for the states 
where the three year total obligation exceeds the three year total allocation. 

A third explanation is simply that the reported figures are incorrect. In 
Florida, the fibres provided by the state bear such little relation to the 
federal allocation that this is the most logical explanation. ,o»T^is judgment is 
strengthened by the knowledge that in Florida the state allocation was used to 
supplement the federall^^ administered Part D project (Sec 14J(c)) and that the 
, sttttc office did not rep<yrt on'Sec, 142(c) expenditures independently. 



' Fifteen of the SEA interviewees indicated that Part D(d) funds were used to pay for staff time devoted te ad mi; 
^rli:g the program, but no relation was found between thl; pr.ctlce *and the ^rcentage of allocated funds 
v>3S oblij?atcd. * • i 
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2r Number of Grantg Awarded * 

The total number of grants awarded in FY 70, 71 and 72, as well as the total 
over the three years combined, is sljown in Table 74. The total number of 
grants awarded in FY 70, 71, and 72 ^ombined was 1, 157,' wiJb a mean of 22, 7 
grants ^er state. ,As can be seen from th^^Offi^tHan and the maximum number 

^ of grants awarded, only a few states awarded a relatively large number of 
grants; as sKown in the state-by-state data in TablV 75, Texas awarded 37 m 

^ FY 70, Wisconsin 27, and the^next most frequent was only 14. Similar <^istri- 
butions aZ-e found in the other yedrs. In terms of the three-year totals, the 
three hi^est States^ awarded aS, 66, and 50 grants, although the median figure * 
. is only 18. Three of the states funde4 only one proj^ct-in-each of the three 
years (the District of Columbia, Georgia, and South Carolina). 



> 












TABLE 74 






TOTAL NUMBER OF GRANTS AWARDED, BY FISCAL'YEAR 




FY- 70 


•FY-71 


FY- 72 


Total 


Number of Grants 


336 


411 ^ 

< 


410 ' 


1,157 . 


Median 


7 


/ 6 


7 


18 


Lowest No. of Grants 


1 


1 


1 


3 


HighestiNp. of Grajits 


37 

> (Texas) 


31 

. (Texas) 


35 

(New Mexico) 


85 
(Texas) 


3.* Number of Grantees Funded : . ' 



The number of different grantees that each stite funde'd is shcavn in Table 75, 
and summary data are shown in Table 76, below. 





TA^BLE 76 






TOTAL NUMBER ^OF- GRANTEES FUNDED, BY FISCAL YEAR 




FY-70 


FY-71, 


FY-72 * 


Total* 


Number of Grantees 


288 . 


388 \ 


' 364 . 


; 642 


Median 


5 




6 


10 


• Lowest No. of Grantees 


' 1 


1 


1 


1 


Highest No. of Grantees 


24 


25 


27 


38 




(Texas) 


(Texas) 

\ 


(New Mexico) 


(Texas) 


^ - 










The total refers to the number of different. grantees and \s thus not equal to the sum of the entries in each fiscal yea'r 


\ 

\ 
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The mea^\ n\imber of different gjrantees funded over the three -^yea^ period 
was 12. 6, with means of 5, 6, 7. 6 and 7. 1 in FY 70>, 7l, and 72 respectively, 
< . As was the case with regard to the number of grants, a few states distributed 

their funds among a large nuniber of grantees, while three utilized only one 
^ grantee (the $EA) over the entire three-year period. (The upper end of the 
range would be expanded if each of the projects funded in the "mini -grant'* 
programs in Colorado and New Mexico had been counted separately), * 

4, Size of Grants Awarded 



The larg*est and smallest grant awarded by each state in each of three. fiscal 
years is shown in Table 75,' with summary data shown in Table'77, below. 
Again, there is a very wide range betwe^en the state with the largest maximum 



TABLE 77 

LARGEST AND SMALLEST GRANTS AWARDED. BY FISCAL YEAR 


Largest Maximum Grant 


FY- 70 

$131,888 
(Florida) 


FY-71 

■i64, 946 
(Georgia) 


FY-72 ^ 

169, 176 
(Fierida), 


Smallest Maximum Grant 


m — 

12,590 
(Oregon) 


14> 518 
(Maine) » 


16,000 
(NDrtft Dakota) 


Largest Minimul^ Grant 


129,246 
(Georgia) 


135, 93^ 
(Miss.) 


169, 176 
^ (Florida) 


Smallest Minimum Grant 


37 ^ 
(Utah) 


141 

(Vermont) 


120 

(Vermont) 



3- 



grant and the state with the smallest maximum grant. Simila.r findings were 
obtained for the minimum grants. In the case of the maximum grants, the 
largest exceeds tlJe^maHest by mpre than a factor of 10 in each fisjal year. 
The extremely small grants shown in Table 77, aTid'Ikble75, i.e,, those of 
under $1,000, generally represent outlays for a specific unit of skill training 
or evaluation*\f a small component of a project. 



Type of'Graintees Funded 



The 642 different grantees funded during the period FY, 70 through FY 72 were 
classified into the five categories sliown in Table 78, The totals shown in the table 
indicate clearly that SEA's have selected local^'school districts as the pri- 
mary agent to, design and implement projects supported \f/ the state^ controlled 
portion of Part D funds, Iri the aggregate, 59% of the grantees were foiind to ^ 
be local school districts and an additional 12% of the grantees were individual 
public schools or resource centers. This category, together with the category, 
of vocational or technical schools, accounts for over 50% of the grantees funded 
in 44 of the 51 jurisdictions. - 



i these states a large number of small grants were made individual teachers, counselors, etc. , to enable them to 
^implenjent innovations4n their school or classroom. For most purposes, the states report these a%a single block 
^^t to one grantee who admlnistcts the 'Vninl-grapt" effort, * . 

'•'^ ' . 
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I A a Li 
TYPES OF 


:;rantees' 








i 


;Public Sch, Dist, 
Ind, SchyRes.C 


^Voc, or Tech, 
It. School 


f Coniinunity Col 
lege/Umversity 


St » t<. Dcpt» 
j of I »hicau6n 


I Cr ♦ imzat^on 




1 4 1 i '• ' 








/ 


t- 
1 


J TOTAL 


I . \uibama 


6 






1 


1 '. ...! 


t Z. Maska 


" 7? 

0 






-1 1 -^i 


1 d. Arizona 


7 




I 


1 




1 4. ukansas 


4 


. 3 


2 






! 0. ^Lalilnrnia 


18 




7 






left 1 

1 6. I oloradn 


9 




5 


1 , 


' 15 


7. ( onflecticut . ' 


5 


1 








■ ' 7 


o. ih'ia»arc 


, 5^ 


- 1'. 


1 




' » 7 


o ii. >>.^t^ri^ 1 L 
y. Hisifict 01 Lnlumhia 


1 








' 1 


in I 1 ' 1 


2 




2 


1 


1 5 


11. Georpia 








1 


' 1 


1 II 

IZ. hawan 


1 




6 




7 


luuhn 


7 




1 




! 8 


1 4 III* 

14. iMinois 


16 




> 4 




i ' 20 


IZ l_J « 

Id. Indiana 


8 


- 


I 




; 9 


l*v ' 

; lb. Iowa ^ 


2 




2 




4 


|W. Kansas { 

10 L > 1 / 


; 3 


3 


1 




7 


lo. KcnliicKV • f 


4 . 








' 4 


ly. Loui<Mana 












20. Maine 


1 17'^"' 


, 2 


2 


1 




Zi. Maryland , 


10 




4 




; 23^ 

t 14 


22. Massachusetts 


' 5 


2 


1 




^ i 8 


23. Michi;:an 


\ 6 








i 7 


24. Minnesota 


18 




1' 




; 23 . 


25. Mississippi 


/ » 1 






1 


! 2 


Zf>. Missouri • 







- 1 1 


3 


£i. >iOtitana 


/ 12 


1 


' 1 ^ 




} 14 


<iOv .>eorasKa ^ 
JJo \J_. _ J,. 


~1 ; 

V 3 




2« 






; S 


<£7. .\e\ada 


5 




1 




; 6 


30. Ncu Hampshire 


10 


1 




1 


i 


t 13 


3t.- .New Jersey 


11 , 


4 




1 




• 16 


32. Ne^v Mexico 


26 • 


2 ' 


4 




I , : 33 


, o.}. .>ew i ork • 


6 










, 7 


♦>4. .lortn Laroiina 


6 * 


9 


8 




j h— 

! 23 


*K>. i>ortn Uakota 





* 


3 




> 10 


Jo. Unio 


13 ' 


* 






' 13 , 


Of. Uklahoma 


9 


6 


4 




2t 


.Jo.^ Urcgon 


— ^ — 

17 








; 21 


6y. I ennsyivanta 


6 


3 


2 ' 




• 1 . 


; ^ 

12 


tfi f>i 1 i« J 
4U. nhnde Island 


14 


1 


2 




I ' i 18' 


>oulM L«iro'ina 








1 


. ' 1 


42. ^outll D.ikota 


4 ' 




1 




i 5 


4*^. Tennessee 







• \ 


. 1 


. i ' 


44. Texas 


, •3> 




2 




' 38 


45. L'tah , 


13 


1 

^2 


2 ' * 





. \ "19 


46. Vermont 


11 


3 


V 1 


1 


16 


4*. Virginia ' 


7 r 


— t'^ 






13 


V\a^hincton 


26 




9 




y 


35' 


%e'*5t \ irL'inia 


15 ' - 


1 •> 


\ ^ * 




. 16 


50. ' ttwconstn 


« 11 


' 13 




1 


25 


.51. , Wyoming 


li . 




2 






19 


TOTALS 


454 

(71 r,)^ 


*62 

. (10%) 


(16^^ 


19 


6 












» 
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While the vast majority of grant e>es we^e schools, colleges^ or public educa- 
tional agencies^ it is of interest to note that only six states included grants to 
v.hat might be c^sidered atypical grantees. ' The private organizations funded ' 
included An inde|pindent education agency (Maine), a srocial agency for the 
^ *, physically handtcapped (Alaska), a Health Careers Council (New Hampshire.), 
private employers and the school district (New Mexico), * an Opportunities 
Industrialization Center (Rhode Island), and the* Personnel and^Guidance Asso- 
ciation (Pennsylvania). The selection of«these institutions is quite consistent 
with Part D, Section 142 (d) of the Act^which provides tl^at the state board may 
make grants "to local education agencies or. other public ;or non-profit private 
agencies, organizations, or institutions including business. and in^strial 
concerns. . . as it determines will most effectively carry out the deydop^ent, 
es^tablis^hment and operation of exemplary and innovative occupational education 
programs or projects designed to serve as models for use in vocationSl 
education programs.'" * 



^ 



In 17 states, grantees included State Departments of Education. In two of * 
these states (Arkansas and Oklahoma) separate sfate vocational education 
agencies also, served as grantees, thus raising the total number of Entries in 
this category to 19. In three cases (Georgia, Mississippi, and South Carolina), 
the State Department of Education w^s the only grantee of the siate. In many , ' 
cases, the projects admin^ste red liy the st^te agency grantees encompassed 
^eve raL school districts. In addition, there were several cases in which pro- 
jects which were administered by a Community College or Univers'ity were 
* % operate^ with one or more school districts. In such cases, the close' collabor- 
ation betwee'n the grantee and LEA may be presumed. In one instance (Oregon) 
a community college anjd a high school served jointly as the Part D grantee. 

6, Duration of Projects \ 

Of the 331 projects-f untied in FY 70, 47% (154 projects) were funded for one 
year only; 22% (73 p rojects), we re^ funded for a second year, and the remaining' 
5l% (104) projects) were funded in all three fiscal years, FY 70, 71 and 72. 
The|J data are shown in Table 79, which also show^ the number of states fol- 
lowing each of the possible project duration patterns. As the table^p^indicates, 
in 14 states aU of the FY'70 p^^jects were also funded in FY 71 and FY 72, At 
least one project was funded for all three years in 38 states (75%), and at least 
one project was funded for only one year in 33 states (65%); Only four states 
funded one-year projects exclusively, and only one funded two-yfear projects 
^exclusively. Eig^t^en states continued all' of their , FY 70 projects iox atieast 
one more year. There are just about as many states whose FY 70 projects 
were of oji^-year, two-year, and three-year duratiorf (13) as there are states 
having all FY XS'projects continue for at least two more years (14). These 
two patterns together encompass^ 53% of the states. 
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' - T-ABLE 7 9 

^PATTERNS OF PROJECT DURATION FOR GRANTS AWARDED IN FY 70 



Pattern 



All Projects of 1 ^yea r duration 
All Projects of 2-year duration 
All Projects of 3-year duration 
Mixed: 1-year and 2-year 
Mixed: 1-year and 3 -year 
MixedJ 2 -year and 3 year 
Mixed: 1-year, 2-year, and 3-year 



Total 

% 



4 
1 

14 
,8 

8 
3 

11 
51 



Number of Projects 



1^ 



27 



47 
' 28 

o 

52 

154 
47% 



2 vr. 



27 

6 ' 
35 

73 

22% 



29 ^ 

: 24 

'i 

9 

' 1 04 . 



Total 



27 
5 

29 . 

74 

*52 

15 

129 

331 
100% 



7, Criteria Used in Fuding J^rojects° 
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In order to obtain information 6n the strategies that may have been used in " 
funding Part D(d) projects, a series of questions was asked pertaining to 
various criteria that may have b^en applied in making funding decisions. 
These criteria were: a) type of activity proposed;, b) the annual amount of 
money proposed; c) the duration of the proposed project; d) the t^pe of grantee; 
and e) the type of expenditures. In additrion, the interviewee was asked 
whether the policy with_regard to each criterion e^lement had changed since' 
the start of the, prograrnj^ A summary of the responses elic\ted is presented 
on the following page (liable 80). * - 

All but eleven of the states indicated tliat they used one or more fairly specific 
^^ogrammatic criteria in their funding decisions; fifteen states cited more 
than one programmatic criterion." As shown in Table 80, the most frequejitly 
mentioned criterion was related to the project*s goals in the areas of career 
awareness, career exploration, or career preparation (15 states)^ while the 
remaining criteria were stated in terms of process, audi as te ac he r training, 
or curriculum development, of in terms of the specific tar|;et group, .Tucli as 
disadvantaged students. Eight of the states said they were specifically looking 
for innovative #appr6aches in connection with whatever process factor they used 
as a crite^rion. Half of the states had changed their prjogrammatic criterion 
at some time in the preceding four-year period, almost always in the direction 
of increased specificity. ^ 

bnly nine of the states* utilized the funding level of the project as a criterion, 
and only .fiVe of these had changed this criterion during the period. Five 
other states had changed from using a fundi^ig level criteri.on to not usipg such 
a'criterion. . * . • ' 

» 

Ten states preferred or required that projects Be of one-year duration, and^ 
13 states prefer'red or required longer projects, ,Tvventy-six states had no 
preference or requirement with regard to project duration. Only six of the 
states that used duration as a. criterion had adopted this policy some time dur% 
' ing the four-y,ear period rather than the beginning of the program. 
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TABLE 80 
FUNDING CRITERIA 



Activity^Used^as Criterion 




Career a\5^.a^CTi6ss, exploration or 

p^lpai-ation ' ' ^ I 

Guidance / 

Place rnent ^ . , , / 
Tecfcher Training 



Nuoiber of States 



i 15 
7 
4 
6 



Number of States 

that Changed 
Criterion Policy 



Curriculum or materials devel^opm^nt * '6' 

^Spe-cial Student group (e. g. , ^disadvantaged) 5 

Innovative approaches % ' 8 

f- Other / 5 

Unspecified or No'ne * 11 



Funding Level Used as Criterion * * 

$1^,000 or smaller maximu(m 
Approximately $50,000 maximum 
Approximately $100, 000 maximum 
Specified minimum 
QthWr 

No specific criterion 



2 
1 
1 
1 

4 
40 



\ 



9 
9 
5 
3 
23 



Type of Expenditure Rejected or SeVe rely Limited 

Capital- expen<^iture, including ecpiipment' 28 
Salaries .6 
Other • ' 1 
No specific criteripn ^ ^ 18 



,Oth^r Criteria 



/ 



Did not use other criteria • 
,Used other criteria:^ 

Guidelines, plans, etc. 

Local funJing 
* Dissemination, etc. 

tfeogripmc spread 

Innovatiye 

Independent evaluation ^ 
Newly participating LtAs 

Competitive 

Plans for corttinuationA 
Voc. Ed, Centers 



25 
|4 

6 
4 
7 
1 
2 
1 
1 




■vn4"5" 



7 
4 

4. 
3 
1 
3 
5 
4 
2 



2 
1 

0. 
0 

5 



Duration Used as Criterion^ * » 

* k t 

Three year projects preferred ^8 .1 

Three year maximum • / 3 ' 0 

^ One year maximum or prefer re's, y 10 5 

c Two-three years {prefer red, ^ 2 0 

No specific criterion; projects funded one - 

r year at a timf - ^ ^ - 5^ . ^ 

No specific; criterion; 'funded policy 

unstated' ^ * ^1 ' 1_ 

Granteq Type Used as Criterion 

V 

^^EA only or perferred 

Public institutions only or preferred 
Non-profit institutions eligible 
, Ofiier , * . e 
No -specific criterion 





Almost half <p^ ^t^sta^te^ did not restrict the/types of grantee that might be 
funded; eighteen of the remaining states divided equally into those that pre- 
ferred or required L/EAs to be the grai<?Bfces, and those that preferred or . 
required both LEAs arid ot^er public instt^utionsL>6ave more s'tates expanded 
this criterion \(5 incKide non-profit private institutions. Only nine states h^i 
.changed their^pohyies with regard to grantee type^during the four-year period 

Vpver half c/f the^^^^Jjes' (28) indicated th^t they' r.€fjected the expenditure of funds 
'for capital purposofe*' including equipment, generally on the grounds that such ^ 
^ expenditures wer^^Jegally prohibited. Six ^t ate s rejected salary expenditures, 
and ei-ghteen states .drd not cite any specifier expenditure restriction. Again, 
•few states (7)«h^d ;changed their policy during the life of^tHe program. ' 

^ / *' ^ * y 

TjKventy-fQiii: spates indicated additional criteria: seven stressed plans fox 
^dissejninatiop' or "transportability" or replicabilit^^ six states indicated that 

* the proposals were judged in te rms. 'of co;isDnance with the s'tate plan, guide - 
lines,' or other centrally ba^ed cohsideraJ;ion. Four states required Ipc^ 
contributions!. * , • 

^^1^- Qt)jectives of Projects • ^ / • - 

> SEA repres,entatives were ^sked to indicate which objectives were involved in 
ea^h of the projectsUKat had been funded. By assigjiing the total dollar amount 
of a project to elch of it3» objectives and then compyting for each objective its 
J pe;*centag6 of the total y€?ars funding, one may ai'rive at an approximation of 

• '^the relative i/nportante.* ^fesigned to each of t^e^ mandated objectives. It should 

be noted rihat 'this apprcJach involves the assurnptior\ tjiat.^ach of a project'^ 

objectivefe'isj of approximately equaL importance, "^the astsumption that the va>- 

ious individuals who jpi;c}vified the data on project objectives iitilized similar 
inte.tpretja^ipns of what' each objective encompassed, and the -assump^on that 
the fnforipnants were }crfo\yiedgeable concerning fhe .6b^*ectives of the projects. 
-Sinc^ ea<:hjdf these a^s-umptions is open to question,*, the findings, \yhich are 
summari;z^djin Table; 8f^,| t>elow, must.be viewed indications of gros^ trends/ 
rather than as definitivfe:<Jata. « "t^ 

- ^'^U ^ ' :x : " ^^^^ ; 
i - \ ' ' * ^' ^ : * . . * \ . 

\ 'table 81 . ~' 

PERCENTAGE: OF OBLfQAfTED FUlMDS ATT RIBUTED\TO OBJECTIVES 



1, 

2. 
3. 
4. 
5. 
6. 
7. 
8. 



ObjectiVA^ ^* 

.\\ \ 

Occupational\F\arr\iliarizatit>n • 
oWork Expe ri^n«e \ 
Guidance and' PjV cement 
C v;yr r ic ulum Im p r w me n t 
•Exchange Pel:ao(inel 
Young Workers 
VQcation/dT7'5kuc\tid}^ Careers 
Other 



Year 




ucUtid}^ C 



-; ^ ^ ^ V 

See page 134 for list o£.o^e\:tives ' 

'■ i^^Lj — 




Y\ar''.li ^ Yea 
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Although the figt^re5,shown in»£he t^me suggest that tfhe distribution of lunds^ 
' was extremely stable from year to y^^ . exainination of the data for^^the 

individual states rcvdals a highly variea^icture. Some states sjx'ow a decided 
shift in resource allocation, with some increasing in a category and others 
decreasing in that category^ For exaii^ple, Montana put 45^o of its first year 
allocations into objective 1 (to familiarize students with oo^upatioris ^nd 
requisites), and thereaiCer p*ut ZZ% and 13% into this objeetiv^ \rt^ the two s\^- 
sequent years; New Yorjc' on the x^the.r hand, started out with 7% in this same 
objective and ii;u:reased to ZZ% and^45%*in the next two years.X Further, there 
IS also a wide range from state to state within each of the popikar^objectives , 
such as 1 and 4. For objective 1, the percentages range frohi 0*^0 to 52% in 
Year 1 and from 4% to 85% in Year 3; for objective. 4, the pe]*cen^ges range ' 
from 8% to 1»00% jln Year 1, and from 10%, to 6Z% in Year 3. The fi\\^ihgs tend 
to indicate tJiat many states shifted their priorities from year to ye^, either 
on the basis of an overaM plan which encompassed seVeral objectives^n a 
sequential pattern, or as a result of funding decisions which were bas6!g,on 
considerations other than those ,of proposed project objectives, 

9. Program Operations A 

\^ 

The relation between the SEA and the state-funded Part D(d) projects carried 
out by LEAs was reported to be^iiite extensive, as shown by the figures' inV 
Table BZ^Almost all of the SEAs indica^ted that "they monitor, provide techV-,^ 
meal assis'Kj^ce, disseminate project information, and provide services up»oK^ 
request. Almost all of the SEAs reported contact with the LEAs at leas^t three 
time.s per year, and 40% indicated that the contacts occiirred at least monthly. 
Since the responses to the cjuestions on role and frequency of contact show 
. little variability, there is clearly no relation between, these data and other 
variables, such as grant purposes, or number of afi:ants funded. 



TABLE 8Z 

^ ROLE OF SEA^JN PART D(d) LEA PROJECTS - 



Role of SEA ' Numfeer of SEAs 

Monitors projects ^ 49 

^Provides technical assistance 47 

Disseminates project information - 48 / 

Provides services upon request 49 / 



. FREQUENCY OF CONTACT BETWEEN SEA AND PART^Xd^ LEAs 
Frequency of t aotact , Numbe/of SEAs 



At least monthly 
Three or more pof year 
One or two per year 
As needed 



/ / • 

Four of the SEA interviewees indicated that in .their opinion. all of the state 
funded Part D projects were suitable for replication, and the, remainder 
indicated that "some" project^ were suitable. Thirty-six interviewees 
specified t^e particular projects which in their opinion contained activities 
that were suitable for replication. The total number of such project^ cited 
was 144 (4 per interviewee). The total nvimbe r of projects in these'36 state.s 
since the start of the progra,m was 601, yieldi/ig the finding that 24%; of the 
potentially replicable projects we re Specified as being replicable /at least 
in part. (If the four interviewees who said that all of th^project£ in- their. 



btates were suUable for replication are included, the total number of pro><!xts 
which^ould havb been cited was 68Z, and the percentage of those cited 
"^(^increases to33%, ) It should be noted that in several of these states the inter-' 
Vicwees qualified th^iir judgments by noting that they were not familiar with 
all' of past, or in som^ cases, oh<^oing, projects. In the cases where the 
^ interviewee failed to specify projects suitable jfor replication, typically this 
due to the^ir expressed un familiarity \Mth thd details o'f project operations. 

".T^he interviewees reported a wide variet'^^ of activities i*elated to dissemir>ation 

information and enc ou^cagemertt of repi>i'c^tion, Sorne form bf written com- 

"n\unication, such as newsletters, guideline's, re^jorts, or abstracts, was 

utilized hyf\9 states. Conferences, worksjiops , *or oral reports were reported 

by 17 stsrees, and the use of visits* to der^nstration cx^nters or resource centers 

'was redorted by five states. Twenty -two states a:ep9rte^ that they used more 

than ofQ type of activity and only four reported no activity. Eight states 

eithe r\provided direct fujid's for replication 9r w»e re, actively seeking legislation^ 

authorizing such funding/ . " -'^ ' 

r 

10. Research Coordinating Units and Part D ;^ ^ 
c ^ 

In addition to collecting information on the state admiriiste red Part D funds, 
interviews were held with representatives qf the Research Coordinating Units 
(RCU^s) in 43 of the states vis^ited. The RCUs receive Federal support through 
Part C of the Vocatipnal Eduealion Amendments pi 1968, The purpose of the 
interviews was to Qollect information on the RCU's functions^ roles and respon- 
sibilities in generalj apd more specifically ,**relative the the Part D program. 



V 



.The most frequently cited RCU function was dissejnintipn of information to 
schpol officials throughout the state who were either operating career or 
vQcationaL^ducational programs or were making plans to <fo sp. This service, 
which was cited by 22 RCU^, consisted mostly of dissemmation of descriptions^ 
of on-'-goirig projects irl th£ state and to a lesse.r extent/ of i^esearch findings 
an<^ curriculum guidelines andNi^ate rials , Tl^e research and th§ curriculum 
information was generally deriye"?^om both the' on-going work in the state 
and from information obtained through lite;Tature searches. The proced\:^res 
followed i^i obtaining information and the methods used for diss-^mination Wer^ 
highly variable. The most frequently cited dissemination techniques were 
monthly newsletters,' pa rticipaqion in planning conferences, and direct inputs 
to teacher workshops. ^ ^ 

The second most frequently cited function of RCUs was the coordination of demon 
stratiqn and/or research pr3jects and the State Department of Education. This 
involved ttie transmission. of information to various local a^nd state officials. 
ThiSv^^e of c oordination \vas cited as ark important func tion by 20 of the ilCUs, 



uiMuiuns 



Other frequently dmentioqed fuiK*i?TOns were conducting research (9), collecting 
-and analyzing information useful for project planning or implementation (9)i 
evaluating, reviewing, .or monitoring proje^cts ('iO), assisting in writing pro- 
posal's or planning project strategies (IJL), and providing projects with technical 
. assist^rlcje of various kinds"(IO), 

' V r • ^ 

The nu'mbpr of functions cited by the RCU's varied fyom one (information 
dissemination) to all of the above mentioned function?, and the size of the 
* RGU staffjvaried accordingly^! The state with only one function cited was 
staffed by two people, at 25fJ time each. The most active RCU had a director, 
coordinators for research, for planning, for evaluation, and iot information 
se rvicesj . and^ eight'/othe r profes'sionah staff mem'bers • 

" ■ ' V' 
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The distribution of functions with regard to Part D projects was very similar^ 
tojthose performed overall, except that the second jnost frequently ttted func- 
tion, after information dissemlfiation, was that of providing technical assistance' 
to the projects. Assisting in project planning was cited relatively infreqtiently, 
Only 27 Rtl^'re ported any funct;ion in^ relation to the specific federally 
administered project visited bV DA field staff. s ' * * 



Summary and Conclusions 



An analysis of the' preceeding findings "^as made in an effort to ider.tif-i^nritcerns or 
clusters of states which would permit'inferring the general objectives and strategies 
of the Part D program. This effort did not produce meaningful result^.' For • ^ 

example: ' ^ 

In comparing the categories reflecting percent of obligation with the one 
indicating use of SEA as grantee, ^one would expect to find a high percent 
, of cases in which 100% was obligated. This is most certainly what one 
would expect in those cases where the SEA was the only grantee. The data 
do n'ot bear this out, however. The two states that funded only the SEA 
obligated 86% and 65%, and only six of the 16 states that funded the, SEA in 
addition to other grantees 'obligated lOO^o of their funds< 

^ A^comparison of the project dur^^on pattern with the percent of obligatii 
^^Iso failed to produce a pattern. Of the" 14 states that funded oaly 3 -year 
p^-qjects in FY 70, all but two obligated over 96% their fiSids in that year, 
and\ll four of the states that funded onlyX-year projects obligated at least 
99% ortijeir funds. / 

• , The responses concerning project duration as a criterioain funding were 
scrutinized in* relation to the practiced followed wLtn\^spect to the length - 
of project funding, but no pattern was |ound. The ^ate^^ that indicated a * 
preference for one -year. projects (with no reported change in this prefer- 
ence), and those that indicated a preference for t^ree-year projects, hacf* 

' vefy,, similar distributions in 'practice: Only one of the one-year preferred 

• *group funded its 1970 projects for only one year, and only two cTf the three- 
year-preferred groups continued to fund projects for three years. Of the 
14 states that had funded three -year projects only, nine indicated thc^'t they 
did not use project duration as a criterion in funding., 

• A comparison was made between the states which only funded projects for 
three years and those which only funded J. 970 projects for one year, in 

V ' terms of the proposition of funds spe.nt on each of the eight project objectives. 
It was found that in two of the three "one -year-project^' states for which 
data were available almost all of the funds were allocated to objective four, 
curriculum improvement, whereas in only two of the 14 "three-year" states 
were as much as 50% of ;the Junds allocated to this objective. In contrast,, 
the three -year states seemed tp focus on objective Qhe, fan;Liliarizing 
students with careers and requisites. Apparentl^^, one -year projl^cts tend 
to be planningkcd development projects, while three -year projects tend to 
be classroom implementation projects. ' <u • * 

6ix states and the District of Columbia combined some or all of their Part 
D fi^ds with those oi the federal shar e project^(Part Dfc)). The states 
were ^r}can3as , Florv^, Kans^f^ MT^'si^ippi, Nevada and New Jersey. 
!ln six of the seven case^r-^th^exception b(^i^ng Nevada, the SEA was the 
grantee for the "federal share(^k^^roj%ct. In Mississippi and fe^e District 
of Columbia all of the "stafe sliarc" Part D funds were combined with the 
"federal share" funds. ' In the other five cases, the ^states funded other 
projects as well. In tw^o casps, Mississippi and Nevada, -the amount 
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delegated over three years was just under 80% of the state allocation, while 
in the other cases it was over*90%. In all seven cases the projects received 
, "state share" funds for three years (the length of "federal .share^' projects), 
but the level of funding by year varied across the projects. 

In concluston, the various analyses ^^v^ not produced clear patterns from which to 
infer any distinct set of strategies. Typically, states have granted Part D fundsx^o 
local school districts or individual local schools, The princip^l objectives of the 
funded proje^cts have been to familiarize students with occupations and, to improve 
vocational education curricula, followed by work experience programs and guidance 
and placement activities. In any year, a typical state may be expected to make 
grants which 'vary widely in size and t;o projects which vary in duration from one 
to three years. 



In general, the wide variability in funding approaches and the shifts from year to 
year suggest that the state departments of education in their funding of Part D 
exemplary projects are continuing to search for effective methods fo r. implementing 
the new and inApvative approaches called for by the federal legislation. It appears 
that the problem of identifying the most effective methods of achieving the objec- 
tives of Part D in local school districts has not beert'resolved. 

s 
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CHAPTER Xfl: S^TUDY SUMMARY AND CONCLUSIONS 



Introducticn 
X . 

\ 

Vv'iHi Part D of the 1968 ameixdments to the Vocational Education Act, Congress 
^r.od for the funding of exempfeiry projects which would have as their ultimate 
ii'ial the reduction of the rate oJ^youth unemployment. Congress stated that this ^ 
goal was to be accomplished by|:r^ating- a "bridge between school and earning a 
living." MoVe specifically, th^egisHition author,ized the expenditure of funds for 
exemplary projects designed to^roaden occupational aspirations and opportunities 
for youtlj. i ■* 

The funds appropriated fox the Implementation of Part D were to be administered 
in part by the states and territc^ies and in part by the'U.S. Office of Education. 
Each state was allocated a por^n of the Part D appropriation in accordance with 
a formula set forth in the legislation. During each of ^e first three years of the 
program, the amount of federally^ administered Part D funds allocated per state 
was in the range of $100,000 tc^$200*, OOO^for most states; an equal amount \yas ' 
allocated to be administered by| state government. 

Operationally, federal responsibility for administration of the Part D program w^as 
vested in USQE's Division of Vocational and Technical Education (DVTE). Between 
June 1970 and July 1972, DVTE awarded a three-year grant to organizations in 
each state .^territory, and the District of Columbia for the implementation of Part 
D project^,% The grants made during thi3 period were considered the ''first round|^' 
effort of the Part D program. Since that time, additional grants have been awarded 
under a modified set of policy guidelines which are considered the "second round'' 
projects. In Fiscal Year 1976, a new set of grants, with further modification of 
focus, is planned which will support'^^ "third round" of projects. 

This report represents an assessment'of tH'e federally administered furst round 
projects and a descriptive overview of the state adxyiLnisitered portion of the Part D 
program. The focus oi the assessment has. been on "determining the extent, to which 
the efforts undertaken during the first three years of operations achieved the intent 
of the- enabling legislation. At the start of the study, measures were agreed on 
which would serve as indicators of the extent to which the progi^am and its individ- 
ual projects achieved st;udent outcomes whicji the legislation and USOE policy state- . 
ments indicated were expected. It was determined at the outset that it jclearly 
would not be possible, and probably would be inappropriate, to attempt to assess 
the relationship between the program and the long term'legislative goal of reducing 
youth unemployment. x* * • 

Prior to sumrnaxizing the findings p^resented in the preceding chapters and-drawing 
such conclusion^ as they permit, the positive program management actions which ^ 
USOE has taken on the basis of this study's preliminary findings and^ other reviews/^ 
of the experience should be noted. Specifically, concerted efforts have been made 
0 to be^more specific with respect to the definition of key t^rms, and the student l^vel 
outcomes cfXpected. ^In addition, a ii\ajor emphasis has been placed on improving 
the quality and practical utility of individual project evaluations. These USOE 
actions address the overall" study finding that projects were typically not well defined 
yin terms of purpose or clientele and that this lack of clarity may relate to tlje failure 
in many projects to identify student outcomes significantly related to project activi- 
ties. Thus, the effoj;ts at thifcational level dui^ing the last.year to be more specific 
with respect to both program objectives and managerial practices 'represent explicit 
attempts t(\improve the progran^ in* the years ahead. • ^ . ^ * 
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In large measure because of the explicit intent of the federal program managers 
to gain from the experience of the first round projects, " the ^h^us*^ of this study has 
been un Dbtaining information >vhich would be o^ utility in planning for the future. 
Thus, while in one sense the study may be considerecian evaluation of the perfor- 
mance* of the program with respect to achieving the ifitent set forth in the enabling 
legislation, in another sense its intent has been to present findings with^respect to 
several sets^ of program indicators which wilLassis t both Part D and the newly 
created Office of Career Education in their efforts to maximize thie positive imAacts 
of their programs in the future. ^ ' • ' 

B. Summary of Find ings ^ ' ' • 

The -intent oS^the Part T) program was to fund projects which would implement activi- 
ties expected to assist students in obtaining satisfying employment. Chapter V of 
this report presented findings which related the amount of federal funds awarded tq 
actual pT-o*ject expenditures. In essence, the grant awards may be considered as the 
planned federal fiscal inputs into each project, the local budgets as th,e projects' 
planned inputs, and expenditures as the actual inputs. A review of the year by year 
input data revealed that in moat projects in no single year did the inputs occur at ' 
the level planned. Taking the three years in the aggregate and considering both 
expenditures and the legislative expectation that funds would be expended over a 
three-year time period, it was found that in 37 out of the 50 projects visiffed (74%) 
the federal dollars expended were belov^^the level planned. _ 

In part, the generally high level of project under expenditures was attributed to the 
timing of^e federal gra^t actions and the first year start-up requirements in m^y 
projects. As indicated in Chapter VII, another ptartial explanation for the^under- 
expenditures may be relat;ed to the nature of thfe-^^enditujce records maintained by 
project managers. Many project directors did not have expenditure data ^^^hich 
permitted them to assess project status in eveVi gross fiscal terms. In addition, 
in the great majority of cases they did not ha.ve information which related expendi- 
tures to any set of project activities. 

From the findings on project activities presented in Chapter VI, it was apparent 
that on an annual basis, and across the three years of program op.erafcftns, the 
extent to which projects carried^ut the activities specified ii> the USOE policy 
paper,* which governed the federally administered projects, varie4 considerably 
aK:ross the 50 , projects ♦ While moat projects reported students in most of the USOE 
required activities at some point during the three years, relat^ely few (26%) 
reported having students in all ac^iy*tles. ^ 

The findings with Vespect to project activities may be explained ^rtially by the 
data reported ^in Chapter VII pertaining to program management, \ review of the 
stated objectives of the 50 projects revealed that in many cases the ab+trHfes 
called for by the policy^paper were not addressed; in no case did a project B/dd.ress 
an objective contained in the legislation but not in the policy paper. A conjparison 
between stated objectives and related activity catfegories indic^ated that in many 
projects there was ho relationship between stated objectives and the performance 
of activities which could be related to these objectives. \ ' , 
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Given the findings regarding expenditures and activities, Hjj^findings with respect 
to student outcomes are not surprising. As reported inr Chapter Vllt, the greatest 
impact of the Part D program was at the elementary school level. In general, the 
pro-am appeared to have had less> impact on students at the 9th grade level; at the, 
12th grade level, impact was less than at the 9th for two sampling groups and 
greater for the other two. 

On a project-by-project basis the impact of the program on students was small, 
with the bulk of favorable outcomes in each student group confined to a small group 
of projects. T'hese "favorable" projects were somewhat different for each s|:udent 
group. Out of 45 projects where comparisons were made between participating and 
non-participating 6th graders on six relevant student Outcome indicators, there was 
a difference in favor of participants for three or more indicators jn ten projects; 
the n^os^: outstanding project at this level reported differences"in favor of ^partici- 
pants on five of tlie six indicat9rs. Of the 42 projects where con^parisons were 
possible at the 9th grade level^ -one pr.oject scored positively on six of the nine rele- 
vant indicators, and ten produced positive results with respect to three o^xaore 
indicators. At the 1^2th grade level, nine indicators of student outcomea^were rele- 
vant for each of the four strata sampled. Students whose participation ^n the program 
\ consisted of having teachers who*utilized concepts and/or materials, made avaiL- 
/•able through the local projects In their regular teaching program (i.e. , infusion of 
career ed^ucation concepts into the school curriculum), demonstrated a significant 
difference from non-participants on six indicators in one project; in five projects 
such differences were found on two or more indicators. In none of the projjects 
where comparisons were possible for stud.ents in the Work Experience, Skill Train- 
ings, or Guidance and Counseling groups was a significant difference between^partici- 
. pants and non-participants founcT on more than three indicators. 
* 

Whije the seaurch for relationships between the outcomes and selected project treat- 
ment indicators reported in Chapter XI did not provide a clear set of relationships 
between activity or treatment indicators and student .outcomes, the review of 
respo:>S€ts of non-participating sixth grade t^eachers suggests that in at least some 
prcyjecrts t)i& inability to measure significant differences between participating and 
non-participating students'may>'relate tp non-participating teachers implementing \ 
career ^'Svication concepts in their classrooms. Given the Part D program efforts 
at the state level described in Chapter XI and other federal, state, and university 
based efforts to stimulate the ^plementation of career education concepts in 
^ schools everywhere, this^may wfell Have been a contributing fatto'r. 

Another potential expianation is suggested by the relatively clear relationship 
which wa^^und' between project expenditures and atudent outcomes. As reported * 
in Chapter jC, \^itlr respect to elementary, junior high, and senio;: high familiariza- • 
"tion activities positive indications of project effects on studentSx^ccur red where 
relatively more project funds were expended on relatively fewer students. 

In summary, the data presl'ented in the earlier chapters showed th^rttypieaiiy 
projects were found to have addressed four components with respect to the federal 
activity areas. Virtually all projects had some level of familiarization activities 
at both the elementary arid junior high school levels. Projects generally devoted 
26% of'-their Part D funds to pay for eleiVientary school activities which* involved 
approximately 2,000 students per project. The students participating in elemen- 
tary activities usually amounted to less than half (46%) the total elementary school 
enrollment of the grantfee^nd accounted for over half of the total number of project 
.participants .(62%). ^ ' , ' 

' Juillor high school familiarization activities nnVt>lved' some 47% of the total junior 
high 'Enrollment of the grant^^ and averaged 1,^400 students ^er project. The 
activitre§. at this level involved approximately 34% 'of the totaltnum^er of project 
participants and cost 29% of tKe 'Part D funds . \ V ^ 
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Senior high school activities in\b Ived. some 650 participants per plroject and - * 
represented approximately 35% of the total high school^enrollment of the grantee. 
High sc*h'ool participants represented approxiiqriately 15^0 of the total number of 
Part D project participants and high school activities accounted for 44% of the total 
project costs* At the high schjol level, the typical project reported participants 
•nn familiarization aclivitie-s^ and one other activity, either work experience or 
job entry skirf training. 

Classroom familiarization activities inv olved 'some 94%*of all project participants 
in the average prcgect. Almost always these activities were undertaken by class- 
room teachei;s on a voluntary bas is. Generally, a coordinator hired by the projfect 
provided suppor^t to these teachers. 

^he typical project employed one coordinator for elementary level familiarization 
activities and one for secondary activities. Participating teachers usually received 
some in-service training in career ^education strategies and were frequently pro- 
vided classroom support in the form of rriaterials and curricula guides. 

•* ■ i 

In some projects, counselors, trained and experiencjed in occupational guidance,^ \ 
■provided-training to teachers so that counseling of this t^e could be ^xfended tb , r 
the classroom. Another common variation in approach was for pToject staff ta' 
make regular visits to s^cted classrooms to conduct familiarization activities. 

With respect to faniiliarizadion activities, it was found that the projects which 
scored highest on student outcome measures involved' fewer participants at a higher^ 
z cost than did other projects. Mor'e significantly, these highest scoring projects ^ 
apparently devoted subs tan ti^ItiiJ^ or resources to initial start-up activities: ^ 
These activities included teacher trainings- planning, and curriculum development. 

Work experience activities, where they occuri*ed, invj)lved an average of 110 
students per year in the average project, typically, work experience activities 
involved 1. 3% of the average project's participants at 18% of the total Part D'cost. 
The average p'roject having work expe rience, a.ctivities employed a work experiei^ce 
coordinator or a placement coordinator. Projects differed considerably .with ^ 
respect to the support provided worl^ experience participants. Some projects 
coupled skill training and work experience for all participants. Counseling and ^ 
guidance were also sometimes included. Other projects simply sought placement , 
* opportunities and provided released time from school. Although projects did not 
indicate that special entrance criteria were applied, analysis of student resRon^ses 
suggest that in general, participants in project supported work experience j^ro- 
grams were les-s academically orienterd than students in work experience pragrams 
not supported by Part D. This may account for the unfavorable results of t}>ese > 
students on the outcome measures. 



Skill training activities involved an average of 365 students per year in those 
projects where they were provided. Participants in skill training activities com- 
prised an average of 2.9% of these projects' total enrollment and cofit approxi- ' 
mately 17% of the average ^total project Part D budget. Training usually was pro- ^ 
vided in traditional skill ar'eas but was generally "intensified." Intensification ^ 
usually meant that students. w^re involved for mor^ classroom periods per ^ay 
over a shorter term than was typically the case. /Again, analysis of student 
responses suggests that participants in this activity areajwere less academically 
oriented than other students tested; this may, account for the \infavorable results of 
these students. It is also important to, note that there was a significant decrease in 
the relative level of support provided for skill training activities and an increase 
in thp relative support provided high school familiarization activities during the 
last year of the program. The relatively high cost of work expedience and skill 
^ training activities may'^have been a factor ir^this third year shift in expenditures. - / 
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It is more likely, however, that *the third -year shift had little to!do with 
the effectiveness of the activities or cost considerations,, but resulted from*other 
considerations. Specifically, the development of the concept of career education 
aiid widespread efforts emphasizing infusion of career eclucation materials in 
rt?cent year's are probable factors. Given the emphasis on ca,ree r xiducation, it 
Qjuld be expected that a shift in priorities emphasizing familiarization activities 
might occur. ^ , '* * < 

The typical project providing occupational guidance and counseling did so by includ- 
ing these activities as an integral part of som^e other treatment (e.g., work^exper- 
ience). Students *participatinjg^n only counseling and guidance activities were 
analyzed in only four p rojectsT While the student results presented suggest that 
this approach was not as effective as others in producing student outcomes, without 
cohsiderably^oj-e expe riencefend data such a conclusion is not considered to be 
Warranted. / * 

The USOE policy paper governrng the first round projects required that grantees 
make provision for the continuation of project supported activities froni ^'^regular" 
funding sources after Part D support had terminated. In 14 of the projects visited, 
school personnel indicated that prpject-.activities either had or would terminate at 
the end of the grant peri(3>d, and Ln 19 other cases they indicated that activities ,wou Id 
be reduced. In eight of the projects, project activities either were continuing after 
the termination of the grant or definite pl\ns existed for the continuation. In nine 
cases, activities had actually expanded after Part D funding had ceased. 

C. Overall Conclusions 



Well over half of the teacPhysrs and counselors surveyed in each of>the 50 sites 
visited indicated that in their judgment it was important to include career education 
iri^ the school curriculum, » From this poitit of view, Ihe first.years of the program 
may be jiidged to have had a substantia^ffect. However, the findings presented 
above suggest that, in general, neither the federally sponsored activities nor the 
federally expected student level outcome^ of the program occurred at the level 
planned.. While a number of reasons for this are possible, the findings suggest 
that the most likely are associated with the general lacKof a set of clearly defined 
objectives, definitions, and managerial requirements and procedures at both the 
project level and" at the federal level. More specifically: 

. • The definition of key terms and concepts were neither precise nor con- 
sistent at either the federal or local levels. For example, students were 
\ identified as project participants because they were ijeing taught by parti- 

I cipating teachers, but what constituted teacher participation varied frorn 

attendance at a two-hour career education workshop to ten or more 
released days per semester for in-servi'ce trainlhg', curricvilum develop- 
ment, and classroom planning. This failure to establish operational 
definitions and categories contributed to the inability of projects to identify 
, ^ with assurance participants in the programs and to the inability o*f the 

* federal level to monitor project efforts effectively. ^ 



Budgets and Expenditure records typically «|^je based ^on "line-item" , 
rather, than programmatic activity categorfl^^Determination of i \ 

Jictivity costs was very difficult. Thi,s difficlWy was primarily a result 
\>i the grant application and award process which did not specify costs by;* 
activityj; only/tn the aggregate. This-was furth^ complicated by no provi- 
sion for administrative costs, which meant that mo9t projects attempted to 
prorate'such costs and attribute them to treatment activities. It is probable 
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that this contributed tg the failure of project directors ,to analyze expendi- 
ture data and'of project ^ff to use budget and expenditure data in the man- 
agenient of projects. In a^ddition, it is probable that this contributed bqth 
to underexpenditures and the failure to engage in expected activities. 

Similarly, USOE did not use fiscal data as? management indicators. , 

J^ypically, the federal man^g^ment, staff did not receive or analyze expen- 
iture data. This contribulPed to the underexpepditur e of funds annually 
and the support of some project^ for periods in excess of the three-year 
funding limit stipulated in the/legislation. \ ' * ' * 

-T4ie evidence strongly indi^/tes that exemplary programs require consid- » 
erable start-up activity And time. Failure to adequately anticipate this 
appears to have resulfoi^^in the inability of projects to meet program expec- 
tations. / 

Generally ,^participants in the projects were exposed more to visitors- in 
their classrooms who discussed careers, and went on more field trips to ^ 
I'earn about jobs, than non-participants. The data suggest that this quaoti- * 
tat'ive difference in- the number of such experiences was not sufficient to 
produce a measur^bie/lmpact on stud^ts. Rather, it appears tHat such « 
activfties need tb be ilStegrated into a well-planned and cpmprehens ive, 
effort. ' 

'The number of different approaches to building a bridge betv/een school and 
earning a living undertaken by the first round projects was limited. FeweV 
than half of the projects had work experience or skill training activities. . 
Many of tlie new approaches suggested in the legislation and Federal Regis- 
ter, such as exchanges of personnel between schools and agencies or busi- 
nesses, were not attempted during the first round. Projects g.enerally did 
little to promote cooperation between public educaj:ion and manpc^er 
agencies. . / . 

TfiQ primary, f^bcus of .r^nd one was elementary apd secondary familiariza- , 
tion'r^l;/dj^^^^ntartlon. Most of the total funds and most of the student par tici- 
panfc'g;^,^X^''^^^g^^4 iif'&uch activities. Work experience and skill training 
effprtffsj;^nded febe;e^>{p and involve limited numbers of students. * 

WheV-^^thjer activities Ax^^r'.e-, in it ia ted, such as alternative higfi schools, they * 
' ' r6lTde'cJ'^">S^fe»lj^^^ i^^^ the school system, expensive, and by design, 

$'^vy\^;&l£^<i:t%'^^^ potential, dropouts or others with special needs). 

To a s\|it. college and non-c'ol'J;©^^' bound students in obtaining employment, 
the USOB p-silioy'^^per indicated^tHat p,rojects should prov)k3e specific train- 
ing in job entiy'Sl^fcil.s to students not previously enrolled in vocational pro- 
grams just prio-r\l;^^he time that they leave school. TheJ'data. appear to 
indlcate'that most^rbj.<cts extended such training only to the W^^-coUege" . 
bound. KM.- * / 

' ^SV ■ -? " ' '* 

The projects typically-=t3i;Li3vi^at assist stu'detrts in securing employpient, 

either during school or«Qp.t)n graduation? Generally, the |5lacement ^ 

activities implemented v^'l^xe essentially referral ^services and little or no 7 

followup was provided. Irr^'generar, neither project §taff nor school pers on-^ , 

nel maintained records of referrals' or placement activities. 
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• %he student^responses indicate that guidance and counseling at the high 
' school level was viewed by students as helpful to them. The data also 

indicate that a great many students did not f^el they had sufficier!t oppor- . 
* tunity to receive assistance from their counselors and they would have 
liked to have increased their contacts. » ' - 

) " • 

The Federal Register indiccfted that applications for round one projects would be^ 
•judged in part upon the projects' objectives being "sharply defined, clearly stated, * 
capable of being attained by the prupose^ procedures, and capable of being measured." 
During the first years of Part D, neither at the federal level nor in most school 
districts was this the case. Th<r inability clearly define federal program ubjec- 
tivos^undoubtedly contributed 'lo many |)f tbe difficult/e^ di^c^ussed in this study. 
. With.mt a specified set of federal expectations the individual third party evaluations , 
could only relate to local project goals and objectives. This resulted in a 
large diversity of evaluation approaches. It is not surprising, tYierefore, * 

thdt these evaluations di<3 not provide meaningful or useful management information 
at tlie federal level for comparing projects. Without^ specific'criteriaJor judging..,^^ 
project progress, the USOE really had no basis for making annual or *tnid-p roject 
grant modificati ons . Most importantly, without such predetermined criteria, 9the 
-USOE had little indic*ktion of what criteria would be employed in the final evaluation 
and so were not able to effectively manage for success. ^ 

t - 

A final point in this regard relates to the whole area of the management 
o/ educational projects. While this was not a management practices study, many of 
the»findings and conclusions appear to relate directly to management is sues . The 
points raised above pertaining to advanced planning, the clarity of program and 
pruject*objectives, and the use of fiscal data are management questicns which relate 
to a)ttainmeH^t of s^dent level outcomes. So too, we suspect, are^the points made in - 
^ the text pertaifting the inappropr;iateness of the job descriptions of project staff, 
failure to maintain records pertaining to placement and other project activities, and 
the limited focus and use of third party evaluations. Based on our observations, 
project staff typically were sincere, hard working, and oriented toward the substan-, 
tive aspects af th^ progr^nj^ Typically, however, they were i^otwell versed in the 
use of sound management practices as an aid in bringing about desired innovations 
in their school districts. Thus', based on the experience of the first round projects 
studied, it appears that a USOE focus on,providing guidance and assistance in the 
area of project management would have been of major benefit to the successful imple- 
mentation of exemplary prog rams . ' 

Such^ c one Lus ions as these are npt uncommon in national studies. While they are 
accurate, they are also somewhat misleading, /or they cannot and do'not'cover the 
pre-prpgram* context of the system of federal funding. Legislative int€?nt is frequently 
less than clear, the time constraints do not usually permit careful planning at the 
federal level prior to^unding of local projects; the federal agencies are frequently 
not sufficiently staffed to permit effective management and, when they are, the very 
question "of the desirability of effective federal management of local educational " 
^programs becomes a<luestion. Thus, as long as inherent weaknesses such as these 
persist, evaluation findings will essentially tend be either negative or ambigucu s 
and ^ecomroendatio^ns will be symptomatic instead of definitive*- ^ " 

.In conclusion,, the USOE and ^especially the program staff in DVTE. who have from 
the outset not only cooperated with this study but also have taken actions based on 
preliminary findings, should be conrunen^ded. It is rare for a national program with 
innovative and ambitious objectives to engage in a comprehensive, irhpapt oriented 
^•.evaluation after only three years of operation. ^While some of the findings of the 
<J study ^ay not^be what one might'have wished, given the complexities inherent in 

implementing the Part D program during its initial years, they ought not to be ' . 

surprising. In our judgment, perhaps the most significant aspect of the study is th*e 
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inference which may be drawn from the fact that it was done, and that actions have 
be^n taken as a result. Clearly, USOE, and the staffjof DVTE in particular , have 
evidenced a commitment to improving both the content and the'^management of federal 
education programs. * <>i ' ' ' ^ 
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APPENDIX A: PUBLIC LAW 90-576 , 
October lf>, 1968 - ' ' 82 STAT^ 1080 \ ; 

» "PART D .-•EXEMPLARY PROGRAMS AND PROJECTS . t \\\ 

• " . ' ^Findings and Purpose^ * 

' ' ^ . 

^ "Sec^v 1^1- The Congress finds that it is necessary to reduce the continuing ' 
'seriously High level of. youth unemployment by developing means for giving the 
6ame kind o'^ attention as is now given to the coHegel-preparation needs qf those 
young persons who go on to col^^e, to the job prepaxati^on needs of the two out 
of three youirg persons who end meir education at oX'.b^fore completion of the 
secondary level, too many of whom face long and bitter , months of job hunting 
or marginal work after leaving school. The-.pUrposes of this part, ttjferel^re, 
ar^r^u ^tirn p>late, through f'ederal financial sup^^U^, ne^v ,ways ta create a bridge 
betiween school and earning a living for young people, who are still in scjiool, 
wh(Xhave left school Either by grsuiuat^on or by dropping ^out, or who are^n 
postaeconjiary programs of vocatroTOl ^preparation, and to promote cooperation 
betwaen public education and manpower agencTes. ^ > \ 

"Authorization of Grants and Contracts « \ 

'Sec/V^Z, There are hereby authorized to be appropj^iated J$15, 000, 000 \ 
for the fisdalWear ending jiune 30, \96^\ $5.7,500,000 fo/^heliscal year ending 
June 30,. 1970\>and $75,000,000 for each of the two succeeding fiscal years to 
enable the Commissioner to carry out the provisions of thi^ part, * 

"(b) (1) From the stU^ appropriated pursu^nt^o t'kis part the Commissioner 
shall reserve svich amount', but nat in excess of 3 pex<centurn thereof, ^s he may ^ 
.determine and^shall allot such amount among Puerto J^ico^ihe Virgin Islands, 
Guam, American Sampa, and the Trust Territories of tH5ts^5^ei£ic^Islands 

according to their re*spective needs for assistance under this part»- ^ 'y- ^ 

"(2) From the remainder of such sums the Commissi^iie-r shall allocate ~ 
$200,000 to each State (except for those provided foP in paragraph. ( 1 ), and.he 
shall in ^addition allocate to each such State an amount which bears the same ratio 
to any residue of such remainder as the population aged fifteen to nineteen, both 
inclusive, iij the State bears to the population of such ag«^ in all such States. 

"(c)^ From 50 per centum of the sornls allotted to each State for the purposes 
of this part, the Commissioner is authorized to make gr^trits to oV contracts with 
State boaVds or local educational agencies for the purpose of stir^julating and 
assisting in the development, establishm'ent, and operation of programs or , 
project^ designed ^to carry out the purposes of this part.. The Commissioner also 
m^y make, in suelx^tate from such sums, grants to 6thei^pubirc or nonprofit 
private Agencies, or'ganizations, or institutions, or contracts with public, or 
privafe^age.ncies, ocgam^tions, o^ institutions, when such 'grants or contracts 
will n>ake a^ especially significant cqntribution .to attaining the objectives di this 

part. . •'^'^XN," ^ ' ' . » ' \ ' 

"(d) The State board may useHjj^remaining .50 per centum of^uch sums for 
marking grants to local educati<^^l a*g^^^les^ or ^ither^^gublic oV nonprofit private 
agencies, organizations, or institutions, ?><^contracts \«^t^^u^ or. private 
agencies, organizations, or institutions inchKliiS^;^usine 

upon such terms and conditions consistent withPfhe provisions of^this part and*wifh 
its State plan approved pursuaiit section 123, as it Q^termines will most .. 
effectively 'carry out the development, establishment, an^ oper^ttiob of exemplary 
and innovative occupational education programs* 3r projects designed to serve as 
models\for use Jn vocational education programs. 



"Uses of Funds 



"Sec. 143, (a) prants or coitfacts pursuant to this part may be made, upon 
terms and conditions "cans istent wiHi the provisions of this part, to ioavr all or part 
of the cost of ^ * * 

"(1) plahning'and developing exemplary prograrr^s or projects such as those 
* djescribed in paragraph (2), or 

"(2) establishing, operating, or evaluating exemplary prograrps or projects ' 
d^sighed to 9arry out the purposes set forth in section 141, and to broaden ' 
occupational as'pirations and opportunities for youths, with special emphasis 
given to youths who have academic, socioeconoi?fric„ or other handicaps, which < 
programs or projects may,^among others, include 

"(A) those designed to familiarize elementary and secondary school 
students with the twoad range of occupations for whicJr special skills are 
required and the requisites for careers in such occupations; 

"(B) prograic^s or projects for students -ot^viding educational experiences 
through work during the school year or in thit sumnfer; ' . \ 

"(C) programs or projects for intensive ^occupational guidance and^ 
counseling during the last years of school and for initial job p'lac^ement; 

"(D) programs*or projects designed to broaden or itnprove ^vocational , / 
^ education curriculums;^ ^ ■ ^ >' , 

' * exchaTnges of personnel between schools and othe'r agencies, 

^^idtjtutions, or organizations participating in activities to achieve the ' 
pux"po^se> af'th impart, inclu*ding manpower agencies and industry; 
, ^ ^-^^-^^-^ '^'^ progr^Tns pV*proJe*ct$ for young wdrkerg released from tiHei'r ^ - ^ 

^ ^ ^ joW'^cjja pjapt-time basis for the" purpose «of increasing their/- educational 
, "^^.^^attadn^TfJ^t^^^ / V * ^ ' r 

"iO^'!pYo^gJ:'ams or pjoject^ at the secondary lev^l to moti;/ate and 
' •5?jo?^i^e\PJ-§P«^f^ssional preparation for ptitehtial teachers fp'r vocatibhal- '/ 

' A ^r^s^iit or contra9t pursuan^ to -this part may^be made only .l^^^ffel;'* 

dommi:ssloner is in the case of grants ox contracts made by bim, or t;ij^r^t5tre' 
board,' fS^e c,ase of grants or contractS/Wade by it, detefm''i;ies^ 



/ 
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/ 

/ 
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) th^t effective proc^^ures will bfe adopJ:fedl>v/gi^'nte'es ^nd cpntractoi 
'oordijiate the dev^lo^rgeT4t--»nd op^'ration^^ 

jects tarried out> m^^'gVants\^ contr^ts.-puxsjiiant torVtfis part, with. 

the appropriate Statc>pl?^ - 

havings the/sSim^or 

. "(B) that^^)"t}ie _ 
enp'ol^led ^'rfnonprofitprivacfe^ sj2^o<?ls^fi?'Jtl^'^are^ to'^e^ served^who^e 
edut^tUynal i!eeds a/e of/thV^y^^vylii6^ihe'pfo^*^m or project involved 
is€o hle^, ^oVision'K^^s midl^j^^^the ^F^ajrticipatton pf such students; 



• ••>•./ 



ito plw^' and^itli othe/ pubkc and private program's ^ ^ 

^i^t^t C5)nsis^^4 -witVth^nupiber/o 
fit'privacfe^ sjr^ogrls^ip'Jt^^^are^'to'Bei served.who^e 



"(C) that effective pQlic^e^^j^J^.pp^edur'^s wtlX-fe-^'^pted whi^ch assure 
that F^dera*! f^ds made^^^^abl^^ parf>;'WiU^fiot-be •comfningled- 

^X^jlth .Sta)e or Ideal rfrntt?^^^ C<; ^ ^' ''^^^ , ^ . ' • ' 

^ "(2) The amc?unt ^^^ilable to a'stat^pur s liaii^ fp, ^^cftion l4i{d) shall be 
available Jor cbli^^cn for'gf ants^l:^c to the State plan 

appA^ved under^"^ct'io'^^l23, for^p$tyin*g all'-o^'^he^i'J^^/ o/ programs* desc,rLbed 
in s'ectioli l42:ld) and section ^^U) dur ing^af^^^ -4nd (he -sutsceedlng; ' . 



'^iscalc-yeai;. 

"(?)'\No gfeni or contact (other^hbaji a grantor cpntjra^t witli a Sfate^ 
. ,.toard.) sh^rp;r-made by'*the Commissi/fner under ^jebtion l'42(c) with, re^spect 
, to "^any Dr'ogi;,^i^V project ifnles^ sucb.program-arrproject has Ipeen submitted 
'to th^ Stat^^oac^ in the Sfate in j^fiiph it is tto be conducted and.hds not been 
4isappro^^ by the State bbard withil^fsixty days of.such submla^ion or v^ithin 
. setcjf^Qp'^r period of time as ^the Commissioner may detetmme pursuant to 



' regu^atiins. 
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''(4,) Notwithstanding any other ^provision of law, unless hereafter enacted 
'. o4pr^ss]y in Ur^^yitation of the provisions of this paragraph, funds available 
- ^ 'to- Cornn\ission.^r pursuant to section 142(c) shall remain available until 



expended, f , A . ' 

/ ' " "PayjVients 

'7 * ■ - , 

. H-^y From the amount available for grants and contracts, under this 
part ffursuaS^ta section I42(c ),i in the appropriate State, the Commisffioi^er shall 
pay to ea<f h^^pplicant an amount equal to the^mount expended by such applicant 
in accorda/nce with the approved application, ^uch payment may be made on such 
terms 'a^ are approved in such application. PjayYnent pursuant to grants under 
this pa^'t may be made in installrKX'ITr^ and ij/ advance or by way of ^reimburse - 
meAt;f^'w^h necessary adjustments on a>xie<Int of overpayments or underpayments, 
a.s 't^lre Commissioner may determjine. , 



/ 



"Limitation on Duration of Assistance 



"Sec. 145. Financial assistance may not be given under this part to an 
program or project for a period exceeding three ^ears. 
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APPENDIX B: POLICY SJATEMENT ' ^ 

Department of Health, Education, and Welfare 
Office t>f Education ^ ^ 

Bureau of Adult, Vocational, and Library Programs 

POLICY PAPER AWh V70-1 ' * October 2, /I 969*^ 

* - . . ^ 

SENT BY: Grant Venn, Associate Commissioner, 

Bureau of, Ad^lt, Vocational, and Library Programs # 

, \ 

.SENT TO: ^Executive Officers^f StaleBoards for Vocational Educatioiv; 



State Directors cf vocational Education ♦ ♦ , 

» 

SUBJECT: Highlights of provisions for Exemplary Programs and 

Projects in Voca*tional ^Education . * % • ' * 

INTRODUCTION: The Division of ."^ocatlonai and Technical Education of the . 
Bureau of Adult, Vocational, and Literary Program^ administers' the exemplary . \^ 
programs and projects -under tihe Vocational Education* Act of 1963, as amended * 
by the Vocational Education Amendments of 1968 (Public Law '90-576). ^ Direct 
financial support i« furnished for programs and projects by' the U . S . "Commissioner 
of Education under Sectio^i 142(c) of Part D of-Jthe Act. ^ A. 

Exemplary programs and projects are to be designed to: (a) create bridges 
between school and earnaig a living for young people who are stiU in school, who * 
have left school either by graduation or by dropping out, oi^ w}>6 are in post- ^ 
secondary programs of vocational preparation; (b) promote cooperation between 
public educatid'n and manpower agencies; and (c) broaden occupational aspirations 
and opportunities for youths^ ^iih special emphasis given to youths who have ^ , 

academic,* i ocir>economic , "or Whe r handicaps . Provision is^to be made forfche ' 
participation of students enroll^ in priva*^ non-profit schools. Jlr^pr ojectfs are 
to be conducted under grants or Contracts awar^ed^by the Commis^^fcr, in' 
accordance with the provisions of Part D of the Ac{ and with the appliclble^ Federal 
Regulations. Eligible applicants may^ include local educational agencies, State 
boards fdv Vocational Education, an<^ public. and private agencies^ institutions, 

or organizations. ' , * , ,^ 

^ ^ ■ ^. 

NATURE OF PROJECTS : Exemplary projects represent bridge-bailding efforts 
betweafeii re<3earch and development work on the one hand and actual operations 
in s?^^ settings on the other hajid. Exem^ilaxy projects jvill not involve 
original research i^nd developmental activitt&ff^ut will be based upon.pjj^ior 
research and development. They. will be lirnh^d to^^^artr-reslarch'h^s ^already 
shown will work. Tiwrrlfore, exemplary projects should haye a high probability 
of success. Thfey^hould constitute' a transition of research findings and 
developmental efforts to program^operations. 

FOCUS FOR FISCAL YEAR 1970 : In order to achieve maximum impact, tfie 
funds aj/ailable for fistal year 1970 will be focused on programs or projects which 
combine, intone operational setting, all the following aspects: . - 

1. " Provision for broctd occupational oAentation at the^ elementary Sind 

secondj^ry school levels so as to increase student awareness of tHe ) 
range of options open to thetn/in the \Yo*rld of work. 

2. Provision for! work experience , cooperative educat;ion> ^nd^similar 
J , programs, making possilfle a wide variety of offerings-' in many^ 

occupational areas-* , . * • * , • ^ 
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Provision'for students mi^reyiously enfolle*d in vocational programs to ' 
receive specific training in job entry sRills^ j^st prior to the time that they 
"*leave the school. ^feome of these training pr^'grams might be very intensive 
and of short durati^. ) ' , ^ ' 

4. Provision for intensive occupational guidance and counseling during' the 
last y^rs of school and for initial placemen t of air^ sfudejjts at the ^ 
^completion of their schooling.^ » (Placement might be in a job or in post- , 
secondary occupational training. Placement should be accomplished in 
cooperation with appropriate employment services , manpower agencies, 
etc. ) , \ * ' 

^ 5» Provision for the grantee or contractor to carry the* program on with 
support from regular jfunding sources .after the terjrvination of the ' 
. _ ^ Federal assistance under Part D ofP.L. 9t)-576. (Federal assistance 

under Part D cannot exceed three years* ) 

U^? anticipated tliat other program emphases may be highlighted in future 
fiscal years\» - ^ - • 

FINANCIAL CONSIDERATldNS : Direct grants-and/or contracts will be awarded 
in eaeh State up to the limit of funding available for the Commissioner's use in 
that State as allotted under Part D of P.L. 90-576^. For fi^scal year 1970, it is ' 
anticipated that the amount available for the Comn4ssionef's use for grants^or 
coivtracts^in each State Will range from $100, 000 to $200, 000. Since exemplary 
projects will usually 'requir^ substantial financial resources,' consideration should 
be given to incorporating funds^from a variety of sources, such as transfer funds 
under Part B, Cooperative'' Voc^tioifai Education furids under Part G^. Work Study 
•funds tinder Part H, and funds from State^d local sources. Funds allotted to 
the States^^fpr exemplary prbgrams under a "State plan may be combined"" with the 
Commissioner^' s grant or contract funds into a single Part D project. Generally,, 
proposals which feature a skillful combining of several types of funding, .'structured 
around the central frai;x^e.woijk of a Part D gre^jit, will*be^ viewed with special fayor. 

APPLICATION PROCEDURES : During fiscal year 1970 the cutoff date for receipt 
of proposals will be January 1, 1970. Proposals must be45repared and submitted , , 
in accordance with the publication entitled,' Manual: ^ Instructions and Procedures V 
Exemplary Pitograrftris and Projects in Vocational Education^ Persons preparing / 
^proposals should coJfsuJ^with representatives of their State Board for Vocatfonal 
♦Education regarding> sources of supplementary fading and coordination with 
other vocational education programs and activities in the State. Completed 
proposals are to b^ submitted t9^ the U.S. Office of Education, with copies 
furnished simultaneously to" the applicable* State* Bojard for Vocationaf Education. 
' The*^tat<y Board will »eview each proposal and may, within a period of sixty 
days, disapprove any proposal. All proposals not disapproved by the State Boards 
will bQ reviewed by the U.S. Office of Education,*" on tlie b^si^ of analysis by e 
.^specially-constituted review boards composed mainly of- knowledgeable 
practitioners, The , review will take into eonside ration, su'ch factors, among ^ 
'othePS','/as: (1) the potential of the proposal for^contributing significantly to the 
accomplishment of the purpose of. Part "D of P.L. 90-576; (2) the sound^ss of the 
proposed plan of operation; (3) the adequacy of the personnel and facilities available 
, for carrying.out tKef*>roposal; and (4) the proposal's economic efficiency. State 
Boards' and applicants will be notified of the approMslf or disapproval of each ^ 
proposal, and grants or, contracts will be negotiated with those a^jpUcants whose , - 
\, ^proposals are approved. . - 



Copies of the publication may be obtained by writing to: Division of 
Vocational and 'Techmcal Education, u/.S. Office'of Education, Washingtoti, 
J^'V^-'. 20^02, ' ^ ■ , ^ \ 



\ 



NOTE: ^ ,The above Is ba&od upon the assumptioa that funds will- be appropriated by 
the Congress to activate this program during fiscal yea.r H70, * • ^ 



cc: 



Regional Assistant Commissioners 
Regional Directors of AVLP 
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